Nose Mask Checker

INTRODUCTION

In the wake of this COVID-19 pandemic, nations are putting every measure in place to reduce or stop its spread. However, due to the subtlety of the mode of transmission of the virus, it is almost impossible to detect when it is being transmitted. Also the symptoms displayed by the disease closely resembles that of other respiratory disease out there. Therefore we are left with no option than to adopt preventive measures such as hand washing, wearing nose mask etc.

In most countries, the wearing of nose mask has been made mandatory when visiting places. However there are still a few people who do not conform to wearing them when out in public, putting themselve or whoever they come in contact with at risk. In order to protect institutions/organizations and even homes, we could monitor who comes in and out of the building to ascertain if the person poses a risk. Government institions such as banks and other institutions that provides services also entreat people to wear nose masks when visiting their places of work.

SOLUTION

My solultion involves building a system capable of notifying persons in an office or home when a person who is not wearing a nose/face masks approaches. This to serve as an early warning systems so others can take a plan of action ahead of time. This system will serve as a way of protecting ourselves.

HOW IT WORKS

In this project, a raspberry pi camera is attached to a raspberry (raspberry pi is a pocket sized computer with moderately high processing capabilites. It can be used for Internet of things projects and Edge computing purposes). The camera and raspberry pi combination is installed on the wall or some other vantage point to monitor an entrance.

Tensorflow is a deep learning framework built by google and enables one to build a working deep learning workflow. Using tensorflow, an image classification model is built, capable of predicting between two classes of output images- people wearing nose masks and those not wearing a nose masks.

OpenCV is also another tool used to perform computer vision tasks. It can be installed in both the Python programming language and C++. With OpenCV, a face detection model is built to detect faces in either an image, video or live video stream. Once a face has been detected, it is passed as input to the image classifier which finally predicts if the face is one with a mask or one without a mask.

FFMPEG will be used to stream the live video stream captured by the raspberrypi camera and exposes that stream on a network. Persons in-charge can then see the live stream of their entrance and can see anyone with or without a nose mask on. FFMPEG is chosen because it is lightweight thus making it easier to compile and use on an edge device. It is also easier to use as it does not require a lot of resources to set up.

PROS

• it is small hence can be installed easily
• It runs on batteries thus consumes less power. It can also be run on solar energy using solar panels
• It can be used by virtually anyone

CONS

• It performs poorly at night or low light conditions

One way to curb the disadvantage is to train the image classifier with images under low light conditions. This will enable it perform considerably well in such conditions.
For better functionality, one needs to improve the accuracy of classification model.

INSTRUCTIONS

First off, download the raspbian OS (now raspberry pi OS) from https://www.raspberrypi.org/downloads/raspberry-pi-os/ on a separate computer. There are 3 version you can download with different sizes. Download one that suits you. For this application, I used the raspberry pi OS with desktop and recommended software.

With the OS donwloaded, burn it on an SD card using the etcher software. Instructions on how to use etcher to burn an image onto an SD card be found https://www.balena.io/etcher.

After burning the raspberry pi OS on the SD card, slot it into the raspberry pi and power it with a 2A adapter. Don't forget to plug in your USB keyboard and mouse or better still use a wireless keyboard and mouse. Also, plug in your HDMI screen.
You can checkout this youtube link on how to set up a raspberry pi for beginners

Once you have that lets install some packages you will need.

• You need to install tflite_runtime from tensorflow to enable us run the .tflite optimized file for edge cpu's.
• Install opencv version 3.4 or above. OpenCV >=3.4 comes with the DNN package as well.
• Install FFMPEG and FFSERVER for video streaming.

In your raspberry pi terminal, install opencv 3.4.11 by typing

pip3 install opencv-contrib-python==3.4.11

To install the tflite_runtime for arm systems, navigate to tensorflow.org/lite/guide/python and choose the version suitable for ARM devices. Note that the raspberry pi 3 model b+ has a 64 bit architecture but the OS is only available in 32 bits, thus choose the 32 bit ARM version. You can install this by typing;

pip3 install https://dl.google.com/coral/python/tflite_runtime-2.1.0.post1-cp37-cp37m-linux_armv7l.whl

Once this is installed, you can then install FFMPEG and FFSERVER. Go to this link https://ffbinaries.com/downloads to download. I realized that FFMPEG version 3.3.9 worked very well for this application.
Download for linux-armhf arcitecture. These are binary files an hence can be run directly using bash. To run FFSERVER, navigate to the location of the FFSERVER binary file and type;

./ffserver

This will attempt to run the /etc/ffserver.conf file which can be found in the github repo provided below. Make sure to put the ffserver.conf file in your /etc/ directory on your raspberry pi.

To run the stream and the python code, type in

python detect\ and\ classify.py | ffmpeg -v warning -f rawvideo -pixel_format bgr24 -video_size 640x480  -framerate 1 -i - http://localhost:1994/feed1.ffm

This script receives the cv2 videocapture and transmits it as a feed using ffmpeg. This feed is received then exposed using ffserver and any device as long as it knows the ip address of the rapberry pi can stream the feed in their browser.

BOM