Introduction
Procedure
Hardware Setup
Software Setup
Procedure
Working
Original Research about Face Recognition technique used in this project
Future Scope
Conclusion

Published December 27, 2021 © GPL3+

LDR activated IoT based Face recognition security system

Secure your house with future technology - Face Recognition Security system. Finally tomorrow is here.

IntermediateFull instructions provided5 hours2,551

Things used in this project

Hardware components

Bolt IoT Bolt WiFi Module

LDR, 5 Mohm

USB-A to Mini-USB Cable

Buzzer

LED (generic)

Breadboard (generic)

Software apps and online services

Bolt IoT Bolt Cloud

OpenCV

Microsoft Visual Studio 2017

Hand tools and fabrication machines

Servo Motor, Premium Male/Male Jumper Wires

Premium Female/Male Extension Jumper Wires, 40 x 6" (150mm)

Story

We all have a smartphone nowadays, with face unlock features. But many of us are still using the basic old mechanical locks to protect our house, safe locks, etc. Wouldn't it be amazing to have the feature of a face recognition system installed in our house just like we see in movies? Here, in this project I am going to demonstrate a prototype of such face recognized security system with all information about the process as well as the coding part and the hardware equipment, you may need to build such a system.

LDR activated IoT based facial recognition security system

First, let us see what hardware and software are required for this project.

Introduction:

Now as we know what are the requirements for this project. So, let's see what is going to happen in this project.

The first device the user is going to interact with is our LDR - Light Detecting Resistor.

The face recognition won't be activated until there is a hindrance in the light intensity, otherwise, if we directly connect the face recognition system, it will start consuming memory and power.

Now if LDR observes any hindrance in the light intensity, it will send the information to Bolt Cloud which will be observed by the python code which we have written in our system. And python will start the webcam to scan for authorized personals or intruders. If there is an authorized person in the camera's viewing angle it will detect and close the circuit for port 1 where our relay is attached which will connect the motor to the power source. The motor here will act as a pulley to pull the metallic rod out from its hole. And it will also send a telegram message to the authorities about the access granted with the name of the authority and time. If there is an intruder in the camera's viewing angle, then the circuit consisting buzzer will be closed. The buzzer will start ringing to alert the authorized persons that an intruder is trying to break in. And a telegram message will be sent to the authorities that an intruder tried to break in with the time.

That's was all we are going to do here. Now, let's check out the step-wise procedure to build this project.

Procedure:

Hardware Setup:

Connect one of the LDR terminals to the 3v3 port of Bolt IoT Module and the other to the 'A0' analog port. (This is because the A0 terminal of the Bolt IoT module is the only port available in Bolt IoT for taking analogous input values).
Connect the positive terminal of the buzzer to the '0' digital port and another terminal to the GND port. (This is because the digital port '0' is used to input and output and in this case we are using it as an output port to supply power to the buzzer through port '0').
Connect the LED Positive terminal to the '1' digital port and the negative port to the GND port. (This is because the digital port '1' is used to input and output and in this case we are using it as an output port to supply power to the LED through port '1'). Here, I don't have a relay so, just for the demonstration I am using LED.
Use breadboard to connect these circuits as shown in the diagram. (Using breadboard is not compulsory, we are using it so that we can easily make changes in the circuit later if needed and for simplicity too).
Now, connect the USB cable to the bolt module and another end to the power supply through your laptop or charger.

Here ends the hardware setup. For more information about LDR and Bolt IoT Module click the embedded link.

Note: All these connections are made in the Bolt IoT device, which can be bought from the following link.

Software Setup:

First of all, we need to install python in our system otherwise the "*.py" won't open. You can download python from here. And then make sure your pip is upgraded by entering the following command in the command prompt:

pip -m ensurepip --upgrade
First of all, we need to install python in our system otherwise the "*.py" won't open. You can download python from here. And then make sure your pip is upgraded by entering the following command in the command prompt: pip -m ensurepip --upgrade

Now, you need to install a few libraries to run this project. Otherwise, you will get an Import error. Libraries needed for this project are:

open CV library: pip install opencv-python
NumPy library: pip install numpy
face_recognition: pip install face-recognition
Now, you need to install a few libraries to run this project. Otherwise, you will get an Import error. Libraries needed for this project are open CV library: pip install opencv-pythonNumPy library: pip install numpyface_recognition:pip install face-recognition

For more information click here: Python,face-recognition library, Open CV library, Numpy library, and Pickle library.

Procedure:

Create a python - conf.py: In this file enter the device API and device id.
Second, create a python file - embedding.py: This file is used for creating encodings for authorized persons and create a pickle file for this data to be used for face recognition.
Third, create a python file - recognition.py: This file is the main file consisting of the program to detect the faces for authorized function. And this is the main file that will contain IoT implementation to add functions through our Bolt IoT device.

Working:

First of all, the Authorized person has to create his/her face encodings by just opening the embedding.py file. Then enter their name and any number and tap 's' 5 times to take 5 samples of their face. That's all the encodings will be saved as a pickle file in the same directory in which the file embedding.py is present.
Now when the person wants to unlock the door. He/she has to place their hands over the LDR, which will send a signal to the Bolt Cloud through API that the value of the LDR has been changed.
Now the python code will receive those signals and starts the webcam to start the face-recognition process.
Then the face encodes will be created and compared with the encodings of the previously created pickle file.
Now, if the person is an authorized person, then the signal will be passed to the IoT device that the person is an authorized person and to close the circuit for the port 1 where the LED is connected in our case but relay with stepper motor will be attached in real-time applications. Then a message will also be sent to the authorities about the authority who got access and at what time.
And if the person is not an authorized person then the signal will be passed to the IoT device that the person is not authorized. And to alert the intruder, the circuit for port 0 will be closed that will pass current to the buzzer which will start ringing. And also a telegram message will be sent to the authorities about the intruder along with a timestamp.

Original Research about Face Recognition technique used in this project:

This project is based on the original research published at Modern Face Recognition with Deep Learning. Here the original research states that the face image captured is passed through a deep learning algorithm Hog's Algorithm which follows these steps:

First, the image captured is converted to a black and white image.
Then every pixel is then analyzed according to the darkness of every corresponding pixel.
An image gradient is created in which all pixels are converted to arrows, then the strongest arrows form an image gradient called Hog's Image.
This hog's image of the face is compared with various training to detect the face in face recognition.
Still, this algorithm won't be able to recognize if the face is tilted.
So, now the face landmark estimation algorithm is used to tilt the image to center the lips, eyes, and nose. For this, we are going to use basic methods to tilt like rotation, and scale the parallel lines called an affine transformation.
The basic idea behind this is that there are 68 specific points in our faces created over the lips, outside structure of the eye, top of the chin, etc.
Then an image is created consisting of these 68 points.
Then this image is analyzed to detect faces.
But if an image is compared with several others images every time, it would be time-consuming.
So, now what we do is we create measurements from training images once, and then those measurements are saved. And when the new image needs to be analyzed instead of comparing the image with other training images. We just compare measurements of the new image with previously saved measurements.
Now for measurements, we use Deep Convolution Neural Network to generate 128 measurements of each face.
But still how a computer can recognize these images. For this, we compare 3 images.
The first one is the previously saved image of the known person.
The second is the new image of the known person.
The third is the image of an unknown person.
And if the new image measurements are more similar to the measurements of the previously saved image of the known person. Then the computer will know that the new image is of the known person.
In ML, these 128 measurements of the face image are called embedding.
But to get the accuracy we need to find the best 128 measurements which we could use to get perfect face recognition.
And for getting these perfect 128 measurements, we need to train these images for millions of people, millions of times which is very time-consuming.
So, people at Open Face published their training research. Which is used in projects to achieve accuracy.
This research is used as a reference to create a face recognition library which we use in this project.

Future Scope:

This project prototype can be implemented in our system to add the face-recognition system to it.
This project can be further improved by adding the relay and stepper motor to simulate the unlocking of the door.
In the future, we can add 3d image processing to prevent intruders to use photographs to unlock the door.
In the future, we can add the features to add an image and send it to the authorities to catch the intruder faster.

Conclusion:

So, that's how an LDR-activated IoT-based face recognition security system prototype can be built.

This was just a prototype that's why we used LED here. You can use relay and motor to add the unlocking feature to your project. Furthermore, you can add a telegram messaging feature to this project to alert the authorities about the intruder with a face image of the intruder attached.

Code

import cv2 
import face_recognition
import pickle
name=input("enter name")
ref_id=input("enter id")
try:
    f=open("ref_name.pkl","rb")
    ref_dictt=pickle.load(f)
    f.close()
except:
    ref_dictt={}
ref_dictt[ref_id]=name
f=open("ref_name.pkl","wb")
pickle.dump(ref_dictt,f)
f.close()
try:
    f=open("ref_embed.pkl","rb")
    embed_dictt=pickle.load(f)
    f.close()
except:
    embed_dictt={}

for i in range(5):
    key = cv2. waitKey(1)
    webcam = cv2.VideoCapture(0)
    while True:
       
        check, frame = webcam.read()

        cv2.imshow("Capturing", frame)
        small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)
        rgb_small_frame = small_frame[:, :, ::-1]
  
        key = cv2.waitKey(1)

        if key == ord('s') : 
            face_locations = face_recognition.face_locations(rgb_small_frame)
            if face_locations != []:
                face_encoding = face_recognition.face_encodings(frame)[0]
                if ref_id in embed_dictt:
                    embed_dictt[ref_id]+=[face_encoding]
                else:
                    embed_dictt[ref_id]=[face_encoding]
                webcam.release()
                cv2.waitKey(1)
                cv2.destroyAllWindows()     
                break
        elif key == ord('q'):
            print("Turning off camera.")
            webcam.release()
            print("Camera off.")
            print("Program ended.")
            cv2.destroyAllWindows()
            break

f=open("ref_embed.pkl","wb")
pickle.dump(embed_dictt,f)
f.close()

import face_recognition
import cv2
import numpy as np
import glob
import pickle
import requests
import time,json,conf
from boltiot import Bolt
from datetime import datetime
f=open("ref_name.pkl","rb")
ref_dictt=pickle.load(f)        
f.close()

f=open("ref_embed.pkl","rb")
embed_dictt=pickle.load(f)      
f.close()

known_face_encodings = []  
known_face_names = []

for ref_id , embed_list in embed_dictt.items():
    for my_embed in embed_list:
        known_face_encodings +=[my_embed]
        known_face_names += [ref_id]

mydev=Bolt(conf.api_dev,conf.dev_id)
def read():
        print("Reading LDR value")
        input=mydev.analogRead('A0')
        data=json.loads(input)
        print("LDR value is: "+str(data['value']))
        return data
def buzzon():
        output=mydev.digitalWrite('0','HIGH')
        print(output)
def buzzoff():
        output=mydev.digitalWrite('0','LOW')
        print(output)
def motoron():
        output=mydev.digitalWrite('1','HIGH')
        print(output)
def motoroff():
        output=mydev.digitalWrite('1','LOW')
        print(output)
def send_telegram_message(message):
    """Sends message via Telegram"""
    url = "https://api.telegram.org/" + conf.telegram_bot_id + "/sendMessage"
    data = {
        "chat_id": conf.telegram_chat_id,
        "text": message
    }
    try:
        response = requests.request(
            "POST",
            url,
            params=data
        )
        """print("This is the Telegram URL")
        print(url)
        print("This is the Telegram response")
        print(response.text)"""
        telegram_data = json.loads(response.text)
        return telegram_data["ok"]
    except Exception as e:
        print("An error occurred in sending the alert message via Telegram")
        print(e)
        return False
data1=read()
val1=int(data1['value'])
Found=False
Unknown=False
while True:
        time.sleep(5)
        data2=read()
        val2=int(data2['value'])
        val1=val1+40
        if val2<val1:
                video_capture = cv2.VideoCapture(0)
                face_locations = []
                face_encodings = []
                face_names = []
                process_this_frame = True
                while True:
                        ret, frame = video_capture.read()
                        small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)
                        rgb_small_frame = small_frame[:, :, ::-1]
                        if process_this_frame:
                                face_locations = face_recognition.face_locations(rgb_small_frame)
                                face_encodings = face_recognition.face_encodings(rgb_small_frame, face_locations)
                                face_names = []
                                for face_encoding in face_encodings:
                                        matches = face_recognition.compare_faces(known_face_encodings, face_encoding)
                                        name = "Unknown"
                                        face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)
                                        best_match_index = np.argmin(face_distances)
                                        now = datetime.now()
                                        current_time = now.strftime("%H:%M:%S")
                                        if matches[best_match_index]:
                                                print("Face is found")
                                                name = known_face_names[best_match_index]
                                                message=ref_dictt[name]+", is granted access at :"+current_time
                                                send_telegram_message(message)
                                                print("Welcome",ref_dictt[name])
                                                motoron()
                                                time.sleep(2)
                                                motoroff()
                                                Unknown=False
                                                Found=True
                                        elif name=="Unknown":
                                                print("face didn't matched!!!!!")
                                                message="Intruder Alert!!!!! Intruder tried to break in at : "+current_time
                                                send_telegram_message(message)
                                                motoroff()
                                                buzzon()
                                                time.sleep(2)
                                                buzzoff()
                                                Unknown=True
                                                Found=False
                                        face_names.append(name)
                                        video_capture.release()
                                        cv2.destroyAllWindows()
                                        if Unknown or Found:
                                                break
                                if Unknown or Found:
                                        break        
                if Unknown or Found:
                        break

Credits

Ajay Rawat

3 projects • 7 followers

Electrical Engineering student with a passion for programming. Seek knowledge. Money is just a by-product of our journey toward success.

LDR activated IoT based Face recognition security system