Mask Detection using OpenCV with Bolt IoT

COVID - 19 has not only brought us trouble but also got everyone a reason to learn OpenCV. Also, content for us developers and makers. Face masks have become an important part of our daily lives during the current pandemic. So, wearing them is important for safety and to control the spread. Even tho, the numbers of active cases decrease, it is stupid to ignore the virus that took over the world.

These Masks are the only thing, capable of stopping the spread of this deadly virus, which will help to control the spread. As we have started moving forward in this ‘new normal’ world, the necessity of the face mask has increased. By these means, it is necessary that in addition to using the masks, people use them correctly, always covering the mouth and nose.

In this article, unlike other works, our main focus is to detect whether a person is wearing correctly a mask or not. And then alert accordingly using the Bolt IoT module to the security.

There are some steps to achieve our goal: First we need a dataset i.e. a set of masked faces and unmasked faces. Then run the training model with deep learning on it, which will create a model to differentiate between the two data. Since creating the model takes a lot of time, I have provided a readymade model along with this project. You can also run the training model file to create using the data set given for you.

Step 1: Dataset

The dataset used in this project was composed of images that I found online in published datasets:

• CanalSandeco
• MaskedFace-Net
• Real-World-Masked-Face-Dataset
• My Scraped Dataset

The use of several datasets was necessary to collect different scenarios:

• People of different races and ethnicities
• Masks of different types
• Masks in different positions
• Different Angles

Example (What the dataset overall contains - a bunch of faces)

Step 2: Train the Model (Optional)

In this task, I used MobileNetV2, a lightweight deep convolutional neural network that is small in size and usually faster to process. I used the module, to classify if a person is using a mask or not. I worked with MobileNet implemented in TensorFlow’s Keras API.

from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras.layers import AveragePooling2D
from tensorflow.keras.layers import Dropout
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Input
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.applications.mobilenet_v2 import preprocess_input
from tensorflow.keras.preprocessing.image import img_to_array
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.utils import to_categorical
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from imutils import paths
import matplotlib.pyplot as plt
import numpy as np
import os

I used the above modules and packages to train and build a model. The model will be used directly later. This whole model training process is one time. So, once the model is created, it can be used later on unlimited times to detect the mask

Next, Initialize the initial learning rate, number of epochs to train for, and batch size. The 2 categories that it will differentiate are based on the list.

INIT_LR = 1e-4
EPOCHS = 20
BS = 32
CATEGORIES = ["with_mask", "without_mask"]

Learning Rate: Here, the learning rate is a hyperparameter that controls how much to change the model in response to the estimated error each time the model weights are updated. Choosing the learning rate is challenging as a value too small may result in a long training process that could get stuck, whereas a value too large may result in learning a sub-optimal set of weights too fast or an unstable training process. Epoch: An epoch means training the neural network with all the training data for one cycle. In an epoch, we use all of the data exactly once. A forward pass and a backward pass together are counted as one pass: An epoch is made up of one or more batches, where we use a part of the dataset to train the neural network.

Batch Size: It defines the number of samples that will be propagated through the network. Here, if you have 1000 training samples, then with a batch size of 32, the algorithm takes the first 32 samples (from 1st to 32nd) from the training dataset and trains the network. Next, it takes the second 32 samples (from 33rd to 64th) and trains the network again. Typically networks train faster with mini-batches. That's because we update the weights after each propagation.

Categories: The training is done separately for 2 categories, which is done to differentiate between both. There could be multiple categories to differentiate between, for more processing of the model.

You can go ahead and see the algorithm and code I have used to perform the training. You will find it in the train_mask_detector.py file in the GitHub repository attached to the project.

Step 3: Face Detection Algorithm

The algorithm detects the face, then checks for the resemblance of with mask & without mask model.

from tensorflow.keras.applications.mobilenet_v2 import preprocess_input
from tensorflow.keras.preprocessing.image import img_to_array
from tensorflow.keras.models import load_model
from imutils.video import VideoStream

import numpy as np
import imutils
import time
import cv2
import os

import conf
from boltiot import Sms, Bolt
import json, requests
from playsound import playsound

The first 4 packages are for loading the facemask model. The next 6 models are to stream the live feed of the camera.The last 4 models are for the alert with an IoT platform, Bolt IoT.

killer = 0
check = 0

killer - Count of no. of people not wearing masks
check - Keeps a check every 30 frames (3-4 seconds) if a person is maskless.

detect_and_predict_mask(frame, faceNet, maskNet):
    .
    .
    .
    return (locs, preds)

The above function detects for face, then detects the presence of a mask. The function returns the location of the rectangular blob around the face and the state, i.e. mask or no mask.

frame = vs.read()
frame = imutils.resize(frame, width=400)

Line 1: Starts the video stream with the inbuild camera the device has.
Line2: Resize the frame of the stream fixed to 400px width. To limit the processing efficiency.

for (box,pred) in zip (locs, preds):

This is the loop that keeps looping, detects the face location, and checks for the facemask.

label = "Mask" if mask > withoutMask else "No Mask"
color = (0, 255, 0) if label == "Mask" else (0, 0, 255)

label - Labels on the frame of the face location blob
color - Color of the rectangular blob is green for the 'Mask' label and Red for the 'No Mask' label

label = "{}: {:.2f}%".format(label, max(mask, withoutMask) * 100)

To label a percentage format, how much of it fulfills the criteria.

if check%30 == 0: #run after every 30 loops (3-4 seconds timeframe)
    if mask < withoutMask:
        killer += 1
        print("No mask detected")
        mybolt.digitalWrite(1,"HIGH") #RED Light
        playsound('wearmask.mp3')
        print(".")
        print(".")
        message = "Mask-Less person detected. Total no. - " + str(killer)
        telegram_status = send_telegram_message(message)
        print("This is the Telegram status:", telegram_status)
        time.sleep(2);
        mybolt.digitalWrite(2,"HIGH") #GREEN Light
    else:
        mybolt.digitalWrite(2, "HIGH") #GREEN Light

As stated earlier about the check variable, this part of the loop is run, after every 30 loops (frames). So, that, in case any misread of NoMask is there for a second, it does not misinterpret.

Inside this loop, it checks which label has more value. If the person is not wearing a mask, then in 3-4 seconds the program will print, set the pin 1 to HIGH (in case of any alarm), play an audio file that says "Please wear your mask" and send a telegram message through a telegram bot to the channel directed to. And after a 2 seconds delay, come back to the original loop.

cv2.imshow("Frame", frame)
key = cv2.waitKey(1) & 0xFF

if key == ord("q") or key == ord("Q"):
    break

Next step, to show the overall blob on the frame with every loop to the video stream with openCV.The key function is to stop the loop and come out of it (close the video stream). Here, pressing the 'q' or 'Q' will close the window. Now it's your turn!