Neural network on Ada

Machine learning using neural network is famously popular. In this project, I intend to showcase neural network on Ada.

Usually, Machine learning includes training and inference. This project concentrates only on inference stage. For training, Tensorflow framework is used. Tensorflow framework is used on x86 machine for training on data set. After training is complete, a graph is generated which can be used for inference. This graph basically contains weights and biases for each pixel of input.

Supported layers:

Currently below layers are supported,

• Convolution
• BiasAdd
• Relu
• Maxpool
• Matmul
• Transpose
• Softmax

Training:

For demonstration purpose, I am going to use mnist. Mnist involves classifying 10 classes of numeric digits. More information can be found at http://yann.lecun.com/exdb/mnist/. Each image is of 28x28 pixels with color depth of 1.

To start training on dataset, first dataset should be downloaded. I have used tensorflow script to download and perform training on input dataset. I have added implementation to freeze the graph and store it in protobuf format. A file named "mnist_softmax.py" is used for training and can be found in scripts folder in below github repo. Prerequisite for training is to install tensorflow python package. I have used tensorflow-gpu to run training on Nvidia's GPU. Finally issue,

python3 mnist_softmax.py

This script produces an accuracy of 91% after training. The generated graph is freezed for "serving". This freezed graph contains information regarding tensors present, shape of tensors and tensor values. To obtain these information, "gen_wts_biases.py" can be used. This script prints name, shape and value of each tensor present in the graph.

python3 gen_wts_biases.py output_graph.pb 

After redirecting the output of script interpretation to a file would look something like below,

Source code

Ada source code is present in "src" folder of github repo. Below are the files in "src" folder with their respective description,

• dnn_layers.ads - Spec file for different layers that are supported.
• dnn_layers.adb - Implementation of supported layers are present in this file.
• image.ads - Contains 28x28 pixel array translated object "img".
• network.ads - Top level spec file of network used for digit recognition.
• network.adb - Contains calls to appropriate layer implementation present in dnn_layers.adb.
• weights_biases.ads - File containing weights and biases required by layers used in network.adb

Project video

A word on running CIFAR10 network on STM32F429I. Linker script has to modified to enable external SDRAM for memory consumption. Simplest way is to modify "C:\GNAT\2018-arm-elf\arm-eabi\lib\gnat\ravenscar-full-stm32f429disco\ld\memory-map.ld" to include SDRAM memory space.

MEMORY
{
 flash   (rx)  : ORIGIN = 0x08000000, LENGTH = 2048K
 sram123 (rwx) : ORIGIN = 0x20000000, LENGTH = 192K
 ccm     (rw)  : ORIGIN = 0x10000000, LENGTH = 64K
 sdram   (rwx) : ORIGIN = 0xD0000000, LENGTH = 8M
}
REGION_ALIAS("sram_tx", sram123)
REGION_ALIAS("sram_ro", sram123)
REGION_ALIAS("sram_bs", sdram)
REGION_ALIAS("sram_da", sdram)
REGION_ALIAS("ccm_da", ccm)

Conclusion

This project opens door of machine learning on Ada. This project can be extended to any image classification which opens up infinite possibilities among which medical imaging to detect deadly diseases at very early stage such skin cancer, Tuberculosis etc.,