Traffic sign classification using Raspberry pi and CNN

Hi...

Main theme of this project is to classify and recognize various traffic signs found in roadways. Traffic signs classification is one of the major part of Self-Driving Car system. In the automotive industry, machine learning and deep learning are playing an important role in the development of advanced driver assistance system for improving driver safety. Using Deep Learning it can be developed very easily because it is considered as one of the most emerging technologies. Through this project I'll guide you about step by step development of Traffic sign classifier.

Look at Toyota Road Sign Assist system.

To complete this project successfully you need to know little bit about the neural network construction and basic image processing techniques. Download the dataset from this site. The dataset we'll be using is German Traffic Sign Recognition Benchmark(GSTRB). This dataset contains 43 different classes of images.

To complete this project, you'll need the following packages.

Package Installation:

1. Numpy

2. Matplotlib

3. Keras

4. Sklearn

5. Pandas

6. OpenCV

7. OS, Random and multiprocessing

Load Images

Import all images from your local disk using os and cv2.

os.listdir(path='your_path')

It is used to get the list of all files and directory in the given path. If you don't specify the path it will show files in the current working directoy.

import os

path = 'Location_of_the_dataset'

list = os.listdir(path)

print("Files and directories in given", path, "':", list)

len(list) gives you the number of classes available in the dataset.

Split Data

The dataset is split into training, test and validation sets.

Training data: Data used to train/fit the model.

Test data: Sample of data used to provide an unbiased evaluation of a final model.

Validation data: This data is used to evaluate a given model and fine tune the hyperparameters.

Now, we are going to split our dataset using sklearn train_test_split function. Model evaluation and validation is one of the key aspects of supervised machine learning. Splitting your dataset might also be important for finding common problem such as underfitting and overfitting.

#import from sklearn

from sklearn.model_selection import train_test_split

x_train, x_test, y_train, y_test = train_test_split(images, classNo, train_size=None, test_size = 0.2)
X_train, X_validation, y_train, y_validation = train_test_split(X_train, y_train, test_size=0.2)

Train(22271, 32, 32, 3) (22271, )

Validation(5568, 32, 32, 3) (5568, )

Test(6960, 32, 32, 3) (6960, ) 20% of total images

Here you can see histogram of count of images in each class.

Next, we will apply preprocessing steps to the input images to achieve the best possible results.

Techniques I have used:

1. Grayscaling

2. Shuffling

3. Normalization

4. Equalize

In "Traffic sign recognition with multi scale convolutional networks" paper published in 2011, P. Sermanet and Y. Lecun stated that using grayscale images instead of color improves the ConvNet's accuracy. Let's use OpenCV to convert our images into grayscale.

Grayscalingis the process of converting an image from one color space to another. e.g RGB -> Gray.

Also it reduce the dimension of the given image and its also reduce model complexity. For ex: 4x4x3 image has 48 input nodes and for grayscale input only 16 nodes are enough.

import cv2
path = 'img_location'
image = cv2.imread(path)
gray_image = cv2.cvtColor('Grayscale', gray_image)
cv2.imshow('Converted_image', gray_image)
cv2.waitkey(0)
cv2.destroyAllWindows()

Shufflingthe training data will increase the randomness and variety in training dataset, in order for the model to more stable.

from sklearn.utils import shuffle

X_train, y_train = shuffle(X_train, y_train)

Equalization its helps to improves the contrast of the image

import cv2
import numpy as np

img = cv2.imread("test.png",0)
equ = cv2.equalizeHist(img)
res = np.hstack((img,equ))
cv2.imwrite('res.png',res)
cv2.waitKey(0)

Normalization it is a process in which we change the range of pixel intensity. Its also used to remove noise from the picture. Now we are going to normalize the values between 0 and 1 instead of 0 to 255.

img = img / 255

Image Augmentation

Image data augmentation is a technique that can be used to artificially expand the training dataset in order to improve the performance and ability of the model to generalize. It is supported in the keras deep learning library via ImageDataGenerator class. The performance of deep learning neural networks often improves with amount of data variable. Image data augmentation is typically only applied to training dataset, and not to the validation or test dataset.

datagen = ImageDataGenerator(width_shift_range=0.1,
                             height_shift_range=0.1,
                             zoom_range=0.5, 
                             shear_range=0.1, 
                             rotation_range=90,
                             brightness_range=[0.2,1.0])

************************Model Under construction************************