Drowsiness detector for Drivers

We develop our own drowsiness detector that allows us to efficiently prevent traffic accidents by issuing an audible alarm that establishes the alert pattern for the driver.

What do we need?

We connected our camera to the raspberry pi 4 where the computer vision software will be installed to detect drowsiness.

• Python is the chosen programming language, a version greater +3.7
• Scipy is a python lib for scientific calculations we used the distance module to calculate the Euclidean distance that we implemented in the software logic.

from scipy.spatial import distance

• OpenCV (cv2) is the graphic library that Intel created for the management of images in computer vision. The reading and transformation of the original frames is done with this library.

import cv2

• Imutils It is a python lib that, in union with OpenCV, handles conventions to transform or access arrays of well-defined zones, In our case, it has used to obtain the matrices associated with the eyes after that the face has been detected through the output provided by face detection.

from imutils import face_utils

• Numpy is a python lib that allowed us with its multiple functions to create, transform, store and apply different operations of the matrix algebra to adapt the section or complete image and obtain the result or transformation that the algorithm required.

import numpy as np

• Pygame is a lib created to make video games in python. We used the graphic and sound alerts offered by pygame, it allowed us to display the message on the video output and trigger the alarm sound.

import pygame

• Dlib is a lib with a wide variety of utilities ranging from networks to machine learning, In our case we used pre-trained model for face detection with the haar-cascade model

import dlib

What is the algorithm?

Step by Step

1.- We activate the camera to obtain the necessary inputs, in this case we used the camera of our smartphone to record the video and then send it in.mp4 format to the folder where all our programming logic is.

2.- Through the shape_predictor() and get_frontal_face_detector() functions of dlib and its pre-trained face detection model, we extract the coordinates associated with the face detected in the video.

3.- All the coordinates (x, y) that make up the facial features are obtained as a numeric matrix by means of the shape_to_np() function of face_utils

4.- Once the coordinates of the face was extracted, only the coordinates that define the eyes are filtered.

5.- A new numerical array is obtained with the coordinates that represent the reference points associated with the location of the eyes.

6.- Extracted us the 6 reference points that determine the degree of opening of an eye according to the model of Tereza Soukupová and Jan Čech then we proceed to calculate the eye aspect radius.

7.- Established us a minimum opening threshold in the eyelid, once we reach values ​​lower than the value set in the threshold, we are in the presence of an eye with a high probability of being closed, in our case we set the threshold at Y (we will delve into this later )

8.- We establish a minimum amount of consecutive frames that share the opening threshold of the eye with high probabilities of being closed, in our case we had X FRAMES ( we will delve into this later ).

9.- If the two previous conditions are met together, the alarm is triggered that allows setting the alert for drowsiness danger.

How to know when there is drowsiness?

To determine when the eyes are closed or open, we rely in part on the algorithm proposed by Tereza Soukupová and Jan Čech in their paper "Real-time blink detection using facial marks" the strange numerical coordinates that represent the points that define the beginning. and horizontal end of each eye (Point P1 and P4) and the points that indicate the current position of the lower eyelid (P5 and P6) and the mobile eyelid (P2 and P3), understanding the eye as a circumscribed element on a two-dimensional coordinate axis (X, Y), thus making it easy to handle the relationship of the lower eyelid with the upper one with calculations of Euclidean distance using basic notions of geometry.

Decision threshold

It is necessary to establish a minimum ocular opening threshold to determine when we are in the presence of an eye with a high probability of being closed, to define this threshold we begin to do tests in an interval between 0.1 and 0.5 since there for the dimensions that the eyes usually have oscillate the values ​​that we need to study to find the best fit value with respect to reality. After various tests, we defined as suitable threshold all values ​​<0.3, in this way any value below this number refers to closed eyes and any value above this number refers to open eyes.

Next settings

This threshold worked quite well until we tested with people who have characteristic facial features of countries such as China (very small eyes) or India (very large eyes), adding experiments to test these variants, the threshold was adjusted to <0.2 and quite acceptable performance was obtained.

Number of frames indicating drowsiness

All video is nothing more than a set of images showing one after another at a speed determined by the amount of images that pass in a given time interval, each of these images is called Frame.

To infer about the state of drowsiness of a person it is necessary that they keep their eyes closed for a prolonged amount of frames in this way we will know that it is not a common blink but rather a prolonged closure

For this reason, when carrying out multiple experiments, we determined that the amount of frames that must elapse to determine a state of drowsiness is equal to 50.

Next project