Arduino-Python Drowsiness Detection: Stay Awake!

Drowsiness Detection System Using Arduino and Python OpenCV

in this guide, we will walk you through the process of creating a drowsiness detection system using an Arduino and Python. The system uses a webcam to monitor the driver's eyes and sends alerts if signs of drowsiness are detected. The Python code utilizes OpenCV, dlib and Telebot for the drowsiness detection and communication, while the Arduino code controls an OLED display to show the safety status.

Introduction

Drowsiness while driving can lead to accidents, making it crucial to have a system that detects signs of drowsiness and alerts the driver. This guide demonstrates a solution using computer vision, machine learning, and hardware integration.

Why It Is Important

Drowsiness detection systems are crucial for road safety as they address a significant cause of accidents - driver fatigue. Fatigue impairs reaction times, decision making abilities, and concentration, increasing the risk of accidents. this project is made keeping the truck driver and fleet owner in mind and also here's why it is important.

• Driver Health and Safety: - Long hours of continuous driving, often across vast distances, can lead to extreme fatigue among truck drivers. Drowsiness detection systems provide Realtime alert to drivers, prompting them to take breaks, rest, or perform activities to maintain alertness.
• AccidentPrevention: - Truckccidents due to driver fatigue have devasting consequences. By using drowsiness detection systems, transport owners can proactively prevent accidents and protect both their drivers and the vehicle they operate.
• RegulatoryCompliance: - Many countries have regulations governing maximum driving hours and mandatory rest periods for commercial truck drivers. Drowsiness detection systems assist drivers and fleet managers in adhering to these regulations, avoiding legal consequences.
• CostSaving:- Accidents caused by drowsy driving results in financial losses due to vehicle damage, medical expenses, Insurance claims, and legal settlements. Implementing detection systems reduces the likelihood of these incidents, leading to substantial cost saving for transport owners.
• OperationalEfficiency: - Drowsiness detection systems enable transport owner's driver behavior and fatigue levels remotely. This information allows for optimized route planning. rest intervals, and resource allocation, leading to increase operational efficiency.

Real Life Truck Accidents Due To Drowsiness

• The Tracy Morgan Accident (2014): - In a tragic incident on the New Jersey Turnpike, a Walmart truck collided with a limousine carrying comedian Tracy Morgan and his companions. The accident resulted in the death of one passenger and severe injuries to others. The truck driver had been awake for more than 24 hours before the crash, highlighting the dangers of drowsy driving among truckers.
• North Carolina Bus Crash (2018): - A church bus transporting a group of senior citizens collided with a tractor-trailer in North Carolina. The truck driver admitted to falling asleep at the wheel, causing the accident. The crash resulted in multiple fatalities and injuries, underscoring the need for preventive measures against driver fatigue.
• Truck Collison in Kentucky (2015): - A Semi Truck crashed into a line stopped vehicle on a Kentucky highway. The truck driver stated that he had fallen asleep before the accident. The Collison caused a chain reaction involving several vehicles resulted in multiple injuries.
• Ontario Truck Accident (2016): - In Canada, a transport truck rear ended a minivan that was slowing down due to traffic congestion. The driver of the minivan, along with 3 other passengers, was killed. Investigation revealed that the truck driver had been driving for 13.5 hours, surpassing the legal limit. Fatigue was a significant factor in the accident.
• I-16 Truck Crash In Georgia (2015): - A semi-truck failed to stop to stop and crashed into several vehicles on a Georgia interstate, causing a fiery collision that resulted in multiple fatalities and injuries. The truck driver had allegedly fallen asleep and failed to notice traffic slowdown ahead.

Libraries Used in Python and Their Significance

• scipy.spatial.distance: The scipy.spatial.distance module provides functions to compute various distances between points in space. In our system, it plays a role in calculating the distance between eye landmarks, contributing to the calculation of the eye aspect ratio for drowsiness detection.
• imutils: The imutils library simplifies image processing tasks in OpenCV. It provides utility functions for resizing, rotating, and displaying images. For our drowsiness detection system, imutils enhances the efficiency of handling image frames from the webcam.
• dlib: The dlib library is a cornerstone of our drowsiness detection system. It offers robust facial detection and landmarking capabilities. By identifying facial features, including eye positions, dlib facilitates the calculation of eye aspect ratios for precise drowsiness assessment.
• cv2 (OpenCV): OpenCV, represented by cv2, is a versatile computer vision library that empowers our Python code. It enables various image processing operations, including face detection, contour drawing, and text rendering. OpenCV is pivotal in analyzing the video feed from the webcam and detecting facial landmarks.
• telebot: The telebot library provides a conduit for communicating with the Telegram platform. In our system, it enables real-time communication, allowing the Python code to send alerts and notifications to designated recipients, enhancing the overall communication capabilities.
• serial: The serial library facilitates serial communication between the Arduino board and the Python script. It ensures seamless data exchange, enabling the two components to work in tandem and share critical information.
• pygame: The pygame library enhances our system with audio capabilities. It allows the Python code to play alert sounds, capturing the driver's attention and prompting immediate action in response to drowsiness alerts.

Libraries Used in Arduino IDE and Their Significance

• Wire: The Wire library facilitates I2C communication, enabling seamless interaction between the Arduino board and the OLED display. This library streamlines data transmission, making it an essential component for integrating the visual interface.
• Adafruit_GFX: The Adafruit_GFX library extends graphical capabilities to the Arduino board. It provides an array of graphics functions that enable the creation of intuitive visual interfaces on compatible displays, such as the OLED used in our system.
• Adafruit_SSD1306: The Adafruit_SSD1306 library is tailored for driving SSD1306-based OLED displays. It simplifies the process of interfacing with and controlling the OLED display. This library empowers the Arduino code to showcase crucial safety information to the driver in a clear and concise manner.

How This Project Works

Importing Libraries: The script starts by importing necessary libraries such as scipy.spatial, imutils, dlib, cv2 (OpenCV), telebot, threading, serial, and pygame.

Initialization:

• It initializes the Telegram bot using your API token.
• Sets up the chat ID to determine where to send messages.
• Initializes various parameters like the drowsiness threshold (thresh), frame check count (frame_check), and initializes dlib's face detection and facial landmarks models (detect and predict).

Video Capture:

• The script captures video using OpenCV's cv2.VideoCapture(0) function.
• It then enters a loop to process each frame from the camera feed.

Drowsiness Detection Loop:

• Converts the frame to grayscale for processing.
• Detects faces in the frame using detect and extracts facial landmarks using predict.
• Calculates the eye aspect ratio (EAR) using the provided eye_aspect_ratio function. EAR is a measure of how open the eyes are.
• It calculates the EAR for both the left and right eyes and averages them.
• Draws eye contours on the frame using OpenCV.
• If the calculated EAR falls below the specified threshold (thresh) and drowsiness is not already detected (drowsy_detected is False), the script increases the flag counter.
• If the flag counter surpasses a certain value (frame_check), it indicates that the driver is drowsy. It draws a "Drowsy" label on the frame and sends a message to the Telegram bot asking if the user wants to enable safety mode.
• If the user responds "yes, " the script sends a signal to the Arduino (ser.write(b'D')) to activate the safety mode. It also plays an alert sound using Pygame.
• If the user responds "no, " the script resumes drowsiness detection by sending a signal to the Arduino (ser.write(b'B')) to deactivate the safety mode.
• The flag is reset to 0 whenever the calculated EAR is above the threshold or drowsiness has been detected.

Display and User Interaction:

• The frame with drawn contours and labels is displayed using OpenCV.
• The script listens for the "q" key to be pressed, which stops the video feed and closes the OpenCV window

Drowsiness Detection Thread:

• The drowsiness detection loop is run in a separate thread using the threading.Thread class. This allows the script to perform drowsiness detection in the background while still handling user interaction and Telegram messages.

Telegram Bot Polling:

• The Telegram bot starts polling for new messages using the bot.polling() function. This keeps the script responsive to user messages.

Handling Telegram Messages:

• Whenever a new message is received by the bot, the reply_hello function is called. This function, in turn, starts a new thread (handle_message) to handle the user's response.
• If the user responds with "yes, " the script sends a message to the bot confirming activation and sends a signal to the Arduino to enable safety mode.
• If the user responds with "no, " the script sends a message confirming resumption of drowsiness detection and sends a signal to the Arduino to deactivate safety mode.

PYTHON CODE EXPLANATION

from scipy.spatial import distance
from imutils import face_utils
import imutils
import dlib
import cv2
import telebot
import threading
import serial
import pygame

These lines import necessary libraries and modules for various functionalities. distance is used for calculating Euclidean distances between points, face_utils provides utility functions for facial landmarks, imutils is for image processing, dlib is used for face detection and facial landmarks, cv2 is OpenCV for computer vision tasks, telebot for interacting with the Telegram bot API, threading for multi-threading, serial for communicating with the Arduino, and pygame for audio playback.

bot = telebot.TeleBot("ADD YOUR TELEBOT API TOKEN")

This line initializes a Telegram bot instance using the provided API token.

bot = telebot.TeleBot("ADD YOUR TELEBOT API TOKEN")

Defines the chat ID where messages will be sent.

def eye_aspect_ratio(eye):
    A = distance.euclidean(eye[1], eye[5])
    B = distance.euclidean(eye[2], eye[4])
    C = distance.euclidean(eye[0], eye[3])
    ear = (A + B) / (2.0 * C)
    return ear

Defines a function to calculate the eye aspect ratio (EAR) based on the distances between different points on the eye.

thresh = 0.25
frame_check = 20

Sets the drowsiness threshold (thresh) and the frame check count (frame_check).

detect = dlib.get_frontal_face_detector()
predict = dlib.shape_predictor("models/shape_predictor_68_face_landmarks.dat")

Initializes the face detection model and facial landmarks predictor using the dlib library.

(lStart, lEnd) = face_utils.FACIAL_LANDMARKS_68_IDXS["left_eye"]
(rStart, rEnd) = face_utils.FACIAL_LANDMARKS_68_IDXS["right_eye"]

Defines indices for accessing facial landmarks corresponding to the left and right eyes.

cap = cv2.VideoCapture(0)

Initializes the video capture using OpenCV for accessing the camera feed.

flag = 0
drowsy_detected = False

Defines variables to keep track of drowsiness flags and whether drowsiness has been detected.

ser = serial.Serial('COM3', 9600)

Initializes a serial connection to the Arduino on the specified COM port and baud rate.

pygame.mixer.init()
alert_sound = pygame.mixer.Sound("alert.wav")

Initializes Pygame for audio playback and loads an alert sound.

The rest of the code involves defining functions and utilizing the libraries to create the drowsiness detection system. It's mainly divided into handling Telegram messages, the drowsiness detection loop, and starting threads for drowsiness detection and bot polling. These parts were explained in detail in the previous responses.

ARDUINO CODE EXPLANATION

#include <Wire.h>
#include <Adafruit_GFX.h>
#include <Adafruit_SSD1306.h>

These lines include necessary libraries for working with the OLED display module.

#define SCREEN_WIDTH 128
#define SCREEN_HEIGHT 64
#define OLED_RESET    -1
#define SCREEN_ADDRESS 0x3C

These lines define constants for the screen width, height, reset pin, and the I2C address of the OLED display module.

Adafruit_SSD1306 display(SCREEN_WIDTH, SCREEN_HEIGHT, &Wire, OLED_RESET);

This line initializes an instance of the Adafruit_SSD1306 class, which is used to interact with the OLED display. It specifies the screen dimensions, the I2C communication (&Wire), and the reset pin.

void setup() {
  Serial.begin(9600);
  display.begin(SSD1306_SWITCHCAPVCC, SCREEN_ADDRESS);
  display.clearDisplay();
  display.setTextSize(1);
  display.setTextColor(SSD1306_WHITE);
  display.setCursor(0, 0);
  display.println("Safety Mode is Off");
  display.setTextSize(2);
  display.setCursor(45, 25);
  display.println("100");
  display.display();
}

This is the setup function, executed once at the beginning:

Serial.begin(9600) starts the serial communication.

display.begin(SSD1306_SWITCHCAPVCC, SCREEN_ADDRESS) initializes the OLED display.

display.clearDisplay() clears any existing content on the display.

display.setTextSize(1) sets the text size to 1.

display.setTextColor(SSD1306_WHITE) sets the text color to white.

display.setCursor(0, 0) sets the cursor position.

display.println("Safety Mode is Off") displays the initial text.

display.setTextSize(2) sets the text size to 2 for numbers.

display.setCursor(45, 25) adjusts the position for displaying numbers.

display.println("100") displays an initial value.

display.display() updates the display.

void loop() {
    if (Serial.read() == 'D') {
      display.clearDisplay();
      display.setTextSize(1);
      display.setCursor(0, 0);
      display.println("Safety Mode is On");
      display.setTextSize(2);
      display.setCursor(45, 25);
      display.println("50");
      display.display();
      display.display();
    } else if (Serial.read() == 'B') {
      display.clearDisplay();
      display.setTextSize(1);
      display.setCursor(0, 0);
      display.println("Safety Mode is Off");
      display.setTextSize(2);
      display.setCursor(45, 25);
      display.println("100");
      display.display();
      display.display();
    }
}

This is the loop function, continuously executed:

It checks if there is incoming serial data (Serial.read()) and compares it to 'D' or 'B'.

If 'D' is received, it clears the display, updates the mode and value, and displays the result.

If 'B' is received, it does the same but with a different mode and value.

After each change, it calls display.display() twice to update the display.

Steps To Set-up Telegram Bot

• Create Telegram Account: if you don't have a telegram account, download the Telegram app on your phone and sign up

• Search for BotFather: In the Telegram app, search for the "BotFather". This is the official bot that helps you create and manage the bots

• Start a Chat with BotFather: Start a chat with the BotFather by clicking on it in the search results and then clicking the "Start" button

• Create a New Bot: In the chat with BotFather, send the command '/newbot' BotFather will guide you through the process of creating a new bot.

• Choose a Name and Username: You will be prompted to choose a name for your bot. This is the name that users will see when interacting with your bot. After that, you will need to choose a unique username that ends with the word "bot" (e.g., "myawesomebot").

• Get Your API Token: Once you've successfully created the bot, BotFather will provide you with an API token. This token is used to authenticate your bot and interact with the Telegram API. Keep this token safe and private; do not share it with anyone.

Here you have the bot that you have created

• Now click on START to start the bot

Obtain Your Chat ID:

• Open a web browser and enter the following URL replacing <YOUR_BOT_API_TOKEN> with your API token

https://api.telegram.org/bot<YOUR_BOT_API_TOKEN>/getUpdates

• Now send a message to your bot You will receive a JSON response containing information about recent messages sent to your bot. Look for the chat object inside the message object. The id value inside the chat object is your chat ID.

• Now add the CHAT ID and BOT TOKEN to the Python code

Here is the video Demonstration:

Conclusion:

By unraveling the inner workings of this innovative drowsiness detection system, we've unveiled a symphony of hardware components, libraries, and ingenious code. The system's real-time monitoring, meticulous analysis, and instant alerts harmonize to create a solution that advances driver safety and exemplifies technological prowess. This integrated approach, enriched with versatile libraries, is poised to make a lasting impact in the realm of driver safety, embodying the spirit of innovation that propels our modern world forward.

Precautions for Project Making

• Safety First: Prioritize safety throughout the project. Ensure that all electrical connections are secure, and wires are properly insulated to prevent short circuits or electrical hazards.
• Eye Protection: When working with soldering irons or other tools, wear appropriate eye protection to shield your eyes from any potential sparks or debris.
• Component Handling: Handle electronic components with care. Static electricity can damage sensitive components, so use an anti-static wristband or touch a grounded surface before touching components.
• Proper Wiring: Double-check your wiring connections, ensuring that they match the schematics. Incorrect wiring can lead to malfunctioning or damaged components.
• Software Versions: Use the correct versions of libraries and software tools mentioned in the guide. Outdated or incompatible versions can lead to unexpected issues.
• Test Incrementally: Test your system in stages as you integrate components and code. This helps identify problems early and simplifies troubleshooting.
• Safe Workspace: Work in a clean and organized workspace. Keep tools, components, and wires organized to prevent confusion and accidents.