This page hosts datasets, code and examples for the article on strong neurons.
Requirements. Contour Engine needs powerful CPU for training (GPU code is not implemented yet) with AVX2 instructions and a lot of RAM. MNIST examples can be built on dual-core laptop with just 4 GB, but GTSRB and SVHN require 64 GB of RAM and powerful CPU (non-hyperthreaded 16-24 cores are recommended).
Step 1.
Installing prerequisites (on Ubuntu).
> sudo apt-get install g++ libjpeg-dev
You will need just GCC and libjpeg.
Step 2. Downloading source code and pre-trained networks: contourengine-demo.tar.gz. Source code includes all necessary dependencies: Lodepng image library, ALGLIB numerical library.
The archive also includes pretrained networks which reproduce results from our article, stored using our own binary format in the following files:
pretrained-gtsrb-err173.x1 (GTSRB dataset, test set error 1.73%) and
pretrained-svhn-err471.x1 (SVHN dataset, test set error 4.71%).
Step 3. Downloading datasets in our own binary format (due to large size you may wish to download only ones you intend to use):
These datasets are copyrighted (see MNIST, Fashion-MNIST, GTSRB, SVHN for original references) and are subject to their own licensing terms. We believe that redistributing a copy of these datasets stored in own binary format, solely for the convenience of usage, constitutes fair use.
Step 4.
Building the code.
> cd strongnet-demo
> ./build_demo.sh
This Bash script will build a Linux executable named demo.
Step 5 (recommended).
Disabling hyperthreading.
> echo off > /sys/devices/system/cpu/smt/control
Disabling hyperthreading is strongly recommended.
Active hyperthreading may result in thread oversubscription and/or excessive RAM usage.
Step 6.A. Running MNIST demos.
> ./demo mnist_small
or
> ./demo mnist
These commands will run small-scale and medium-scale MNIST examples -
a training of 10-column Contour Engine network which recognizes handwritten digits.
These examples are the only ones which can be executed on laptop-class system (modest CPU with 4 GB of RAM).
Former needs a few minutes whilst latter needs about 15 minutes on dual-core system.
A network (in our own binary format) will be saved to out/mnist_small.x1 or out/mnist.x1.
Step 6.B. Running GTSRB demo.
> ./demo gtsrb
This demo is a training of 43-column Contour Engine network which recognizes traffic signs.
This example needs 64 GB of RAM and server-class CPU with 16 cores.
On such system it needs about an hour to finish the training.
A network (in our own binary format) will be saved to out/gtsrb.x1.
Step 6.C. Running SVHN demo.
> ./demo svhn
This demo is a training of 10-column Contour Engine network which recognizes digits.
This example needs 64 GB of RAM and server-class CPU with 16 cores.
On such system it needs about 24 hours to finish the training.
A network (in our own binary format) will be saved to out/svhn.x1.
Step 6.D. Running adversarial SVHN demo.
> ./demo advsvhn
This demo is an adversarial attack on the network trained in previous demo.
The pre-trained network is loaded from adv-svhn.x1 file in current directory.
The attack is performed using numerical differentation of the model (in current version),
so it is quite slow - you will need about 30-90 seconds to attack just one example.
The results are printed to console, a copy of original and perturbed images is saved to ./out in PNG format.
