Safety Helmet Detection System using Raspberry Pi AI Kit

Object detection is increasingly being used in everyday applications for a variety of purposes, including safety-critical and precise applications such as safety helmet detection. Working with neural networks used to be a very sophisticated field, but technologies are moving forward with very fast pace. Nowadays, there is many powerful pretrained neural networks models like Ultralytics YOLO and MobileNet SSD.

Until recently, running inference of object detection algorithms on resource-constrained devices like the Raspberry Pi was challenging. The Raspberry Pi is a powerful single-board computer, but it is not as powerful as a desktop computers with GPUs for AI tasks. Recently, the Raspberry Pi Foundation released the Raspberry Pi AI kit, which focuses on running AI applications in an energy-efficient environment. This AI module is a 13 TOPS neural network inference accelerator built around the Hailo-8L chip.

In this tutorial I show you how to do object detection on the Raspberry Pi 5 Board using Raspberry Pi AI Kit. I will convert the custom ONNX model into a format compatible with Hailo hardware.

Prepare the dataset

This dataset contains 5000 images with bounding box annotations in the PASCAL VOC format for the following three classes: helmet, person, and head.

You can download the dataset at https://zdataset.com/free-dataset/safety-helmet-detection/

The following code can be used to convert the bounding box coordinates in the dataset from the PASCAL VOC format to the format required by YOLOv8.

After the data has been collected and processed, the next step is to use transfer learning to retrain YOLOv8 with the new model.
from xml.dom import minidom
import os

classes={"helmet":0,"head":1,"person":2}

def convert_coordinates(size, box):
    dw = 1.0/size[0]
    dh = 1.0/size[1]
    x = (box[0]+box[1])/2.0
    y = (box[2]+box[3])/2.0
    w = box[1]-box[0]
    h = box[3]-box[2]
    x = x*dw
    w = w*dw
    y = y*dh
    h = h*dh
    return (x,y,w,h)

def converter(classes):
    old_labels_path = "./data/annotations_old/"
    new_labels_path = "./data/labels/"
    
    for file_name in os.listdir(old_labels_path):
        old_file = minidom.parse(f"{old_labels_path}/{file_name}")
        
        name_out = (file_name[:-4]+'.txt')

        with open(f"{new_labels_path}/{name_out}", "w") as new_file:

            itemlist = old_file.getElementsByTagName('object')
            size = old_file.getElementsByTagName('size')[0]
            width = int((size.getElementsByTagName('width')[0]).firstChild.data)
            height = int((size.getElementsByTagName('height')[0]).firstChild.data)

            for item in itemlist:
                # get class label
                class_name =  (item.getElementsByTagName('name')[0]).firstChild.data
                if class_name in classes:
                    label_str = str(classes[class_name])
                else:
                    label_str = "-1"
                    print (f"{class_name} not in function classes")

                # get bbox coordinates
                xmin = ((item.getElementsByTagName('bndbox')[0]).getElementsByTagName('xmin')[0]).firstChild.data
                ymin = ((item.getElementsByTagName('bndbox')[0]).getElementsByTagName('ymin')[0]).firstChild.data
                xmax = ((item.getElementsByTagName('bndbox')[0]).getElementsByTagName('xmax')[0]).firstChild.data
                ymax = ((item.getElementsByTagName('bndbox')[0]).getElementsByTagName('ymax')[0]).firstChild.data
                b = (float(xmin), float(xmax), float(ymin), float(ymax))
                bb = convert_coordinates((width,height), b)
                #print(bb)

                new_file.write(f"{label_str} {' '.join([f'{a:.6f}' for a in bb])}\n")

        print (f"wrote {name_out}")



def main():
    converter(classes)


if __name__ == '__main__':
    main()

After data has been collected and processed, next step is to use transfer learning to re-train YOLOv8 with the new model.

Train YOLOv8 on a custom dataset

Creating AI models from the scratch may also be complicated. Luckily, there are many pre-trained models that we can use for different purposes. Transfer learning is a technique to use existing or pretrained model as a backbone of the new model.

I'll utilize the ultralytics package to work with YOLOv8. To install YOLOv8 from pip, use the following command:

pip install ultralytics

Ensure your custom dataset is organized correctly. Images and their corresponding annotations need to be stored in a specific format that YOLOv8 understands

Use the yolo command-line utility provided by YOLOv8 to initiate training. Here's the command for training YOLOv8 with your custom dataset:

yolo task=detect \
mode=train \
model=yolov8n.pt \
data=./data.yaml \
epochs=100 \
imgsz=640
batch=32

YOLOv8 will display training progress metrics, including loss and mean Average Precision (mAP)

When training is done, the model needs to be exported to.onnx format. Models of.onnx format can be run on many major machine learning frameworks.

Export the trained model to the.onnx format using below command:

yolo export model=/home/admin2/Projects/training/runs/detect/train6/weights/best.pt imgsz=640 format=onnx opset=11

Once the exporting are complete, we can proceed to the next steps.

Download and Install Dataflow Compiler

Visit the Hailo developer zone software downloads page: link to Hailo software downloads

Download the appropriate Hailo Dataflow Compiler wheel file for your system.

Open a terminal window and navigate to the directory where you downloaded the wheel file. Install the compiler using the following command,

pip3 install ./hailo_dataflow_compiler-3.28.0-py3-none-linux_x86_64.whl

Clone the Hailo Model Zoo repository:

git clone https://github.com/hailo-ai/hailo_model_zoo.git

Navigate to the hailo_model_zoo directory:

pip install -e .

Convert.onnx model to a Hailo-compatible format using the following command:

python hailo_model_zoo/main.py compile yolov8n --ckpt ../Safety-Helmet-Detection/output/best.onnx --hw-arch hailo8l  --calib-path ../Safety-Helmet-Detection/data/images  --classes 3 --performance

The output will display information about resource utilization during the conversion process

[info] context_0 utilization:
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Cluster   | Control Utilization | Compute Utilization | Memory Utilization |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | cluster_0 | 100%                | 43,8%               | 46,1%              |
[info] | cluster_1 | 100%                | 68,8%               | 59,4%              |
[info] | cluster_4 | 62,5%               | 46,9%               | 43%                |
[info] | cluster_5 | 100%                | 46,9%               | 48,4%              |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Total     | 90,6%               | 51,6%               | 49,2%              |
[info] +-----------+---------------------+---------------------+--------------------+
[info] context_1 utilization:
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Cluster   | Control Utilization | Compute Utilization | Memory Utilization |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | cluster_0 | 87,5%               | 60,9%               | 48,4%              |
[info] | cluster_1 | 75%                 | 100%                | 53,9%              |
[info] | cluster_4 | 93,8%               | 81,3%               | 57%                |
[info] | cluster_5 | 100%                | 75%                 | 58,6%              |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Total     | 89,1%               | 79,3%               | 54,5%              |
[info] +-----------+---------------------+---------------------+--------------------+
[info] Successful Mapping (allocation time: 5m 16s)
[info] Compiling context_0...
[info] Compiling context_1...
[info] Bandwidth of model inputs: 9.375 Mbps, outputs: 4.29382 Mbps (for a single frame)
[info] Bandwidth of DDR buffers: 0.0 Mbps (for a single frame)
[info] Bandwidth of inter context tensors: 17.1875 Mbps (for a single frame)
[info] Compiling context_0...
[info] Compiling context_1...
[info] Bandwidth of model inputs: 9.375 Mbps, outputs: 4.29382 Mbps (for a single frame)
[info] Bandwidth of DDR buffers: 0.0 Mbps (for a single frame)
[info] Bandwidth of inter context tensors: 17.1875 Mbps (for a single frame)
[info] Building HEF...
[info] Successful Compilation (compilation time: 10s)
[info] Saved HAR to: /home/admin2/Projects/hailo_model_zoo/yolov8n.har
<Hailo Model Zoo INFO> HEF file written to yolov8n.hef

The successful compilation will result in a Hailo Executable Format (HEF) file, typically saved to the current directory with a name like yolov8n.hef. The sample model is ready to go.

Install the Hailo Raspberry Pi project. Download the repo of the github

git clone https://github.com/hailo-ai/hailo-rpi5-examples.git

To save the inference results directly to a file, I modified the basic_pipelines/detection.py file by replacing the following line:

+ QUEUE("queue_hailo_display")  + f"fpsdisplaysink video-sink={self.video_sink} name=hailo_display sync={self.sync} text-overlay={self.options_menu.show_fps} signal-fps-measurements=true "

with the following line:

+ "x264enc bitrate=5000 speed-preset=medium ! mp4mux ! filesink location=detection_output.mp4  "

This modification replaces the Hailo display queue with an x264 encoder for video encoding and saving the results to a file named detection_output.mp4.

Then, use the following command to run inference on a video file using the compiled model:

python basic_pipelines/safety_detection.py --labels-json ./safety-labels.json --hef ./yolov8n.hef -i ./test5.mp4

Here's a practical example demonstrating safety helmet detection using the Ultralytics YOLOv8n model, achieving an impressive 30 FPS inference speed.

Detected objects will have bounding boxes and labels displayed on them in real time. There is always a trade-off between accuracy and speed. When working with object detection, it is important to consider the specific requirements of your application and choose the appropriate balance between accuracy and speed.

This guide demonstrates how to leverage the Raspberry Pi AI kit with Hailo hardware to achieve efficient AI inferencing for safety helmet detection. This approach can be adapted to create various applications using object detection networks.

Fianlly, I trained the custom model and tested it using the real scenario video. With the knowledge gained from this tutorial, you can now create and train your own custom object detection models on Raspberry Pi 5 board with help of Raspberry Pi AI kit.

Happy developing!

Acknoledgement

• The video footage used in this demonstration is courtesy of Los Muertos Crew and Akin Victor
• Thanks to the Raspberry Pi and Hailo teams for their amazing work on the Raspberry Pi AI kit, which made this guide possible.

References:

• Retrain YOLOv5 on Custom Dataset
• Hailo Raspberry Pi 5 Examples