Face Recognition Intruder Alert System

1. Introduction

Face recognition technology has wide implications in today's world ranging from governments using it to in public areas for surveillance to it being a security feature on our cell phones. This project utilizes the power of deep learning to build a robust and easily scalable face recognition system that can filter out fake faces ( in photos, videos on a device as well physical photo of the user ).

In this project, I have used my laptops camera as the source of video feed and used my own pictures as data to train the neural network to filter out unknown and fake (spoofed) faces. On detecting an unauthorized face, a buzzer is sounded and the picture of the intruder is sent on my phone via telegram.

2. Demonstration

The trained NN captures the frame and classifies the face ROI as fake/real and sends an alert if fake. Although I have not included in the video, an alert is sent for an unknown real face as well.

3. Circuit and Software Setup

3.1 Circuit Schematic

A buzzer is attached to a breadboard and its positive pin is connected to digital 0 pin of the Bolt WiFi module while its negative pin is connected to the ground pin. If intrusion is detected, the buzzer is switched on and after a certain amount of time, it automatically switches off. I have attached the circuit schematic made using Fritzing.

3.2 Software setup and packages to be installed

I have made this project on Linux Mint operating system. On Windows, you can install a VPS like Virtual Box or VMware to work on using the following links:

https://www.virtualbox.org/wiki/Downloads

https://my.vmware.com/web/vmware/free#desktop_end_user_computing/vmware_workstation_player/12_0

http://releases.ubuntu.com/16.04/ubuntu-16.04.6-server-i386.iso

After setting up, you might want to update ubuntu using the following commands to get all latest versions of all packages which will be better for any project.

sudo apt-get update && sudo apt-get upgrade
sudo do-release-upgrade

• Bolt Python Library

In terminal, type the following commands:

sudo apt-get -y update
sudo apt install python3-pip
sudo pip3 install boltiot

In this project, I have used dlib's state of the art face recognition library which can be installed using:

pip3 install dlib

• OpenCV

Refer to https://www.pyimagesearch.com/2018/09/19/pip-install-opencv/ for installing OpenCV. Install opencv-contrib-python repository as it contains many modules which will be useful in computer vision.

• Tensorflow

Tensorflow is an open source library built by Google Brain team for machine learning applications. Refer to https://www.tensorflow.org/install/pip to install using pip. Remember your entire project should be in the virtual environment you created for installing tensorflow.

Additional libraries like keras, matplotlib, imutils are needed for training and visualising performance of a neural network. The following links might help:

https://solarianprogrammer.com/2018/04/25/install-numpy-scipy-matplotlib-opencv-python-ubuntu/

https://www.pyimagesearch.com/2016/07/18/installing-keras-for-deep-learning/

https://www.pyimagesearch.com/2015/02/02/just-open-sourced-personal-imutils-package-series-opencv-convenience-functions/

• Bolt Cloud

To communicate with the Bolt Wifi module for the buzzer, you need to create an account on Bolt Cloud. It offers features like receiving and storing the data collected by Bolt Modules, Storing the data, Analysing it via Data visualisation and Machine Learning as well as it lets your program your Bolt modules.

Go to www.cloud.boltiot.com to signup.

After creating an account, go to your bolt cloud account and note down the API key which will enable you to communicate with your Bolt device.

• Telegram Python Library

For sending alert on my phone, I have used Telegram python API. Install the Telegram API and refer to this for installing the library on phone: https://python-telegram-bot.org/

Create a new public channel on the app for receiving intruder alert and then create a bot and add it as an administrator to your channel.

Now just add the bot to your channel and you are all set!

3.3 Configuration File

Make a separate python file to store the configuration elements of your device and app:

bolt_api_key = "XXXX"                 # Bolt Cloud API Key
device_id = "XXXX"                    # Bolt device ID
telegram_chat_id = "XXXX"             # Telegram Channel ID
telegram_bot_id = "XXXX"              # Telegram bot ID.

You will have to import this file to your main python file to access the configuration elements.

4. Anti-Face Spoofing Detection and Face Recognition

4.1 Liveness Detection

Various methods have been established and tested to distinguish fake vs real faces. You can refer to this review paper by Chakraborty and Das for more information.

In this project, I have used taken liveness detection as a Binary Classification Problem. A convolutional neural network has been trained on frames from input videos, to distinguish between real and fake faces. You can refer to this blog for in-depth information on this method: https://www.pyimagesearch.com/2019/03/11/liveness-detection-with-opencv/

Since this was a personal face recognition system to authorize only my face, I trained it on sample fake/real videos of only myself. To build a more robust system, you should train it on training data from people of multiple ethnicities. Remember the algorithm and hence the results are only as good as the training data used. Also, based on the level of security you need, you might want to combine different approaches of liveness detection. To train the network, I recorded a 30 second video of myself on my phone for the real dataset and then replayed that video in front of my webcam to obtain the fake dataset.

After training the network, I used matplotlib to get the performance analysis oh how my NN performed on training and validation set:

4.2 Face Recognition

In this project, Face Recognition is a function being called in the liveness detection if the label on the face detected is 'real'. To know more on different types of face recognizers used in OpenCV, refer to: https://www.docs.opencv.org/2.4/modules/contrib/doc/facerec/facerec_tutorial.html

I have used dlib's state of the art face recognition built with deep learning which uses only one user input image to recognize faces (it has an accuracy of 99.38% on the Labeled Faces in the Wild benchmark). Learn more here: https://pypi.org/project/face-recognition/.

The steps this function follows are:

1. Encode a picture using the HOG algorithm to create a simplified version of the image. Using this simplified image, find the part of the image that most looks like a generic HOG encoding of a face.

2. Figure out the pose of the face by finding the main landmarks in the face. Once we find those landmarks, use them to warp the image so that the eyes and mouth are centred.

3. Pass the centred face image through a neural network that knows how to measure features of the face. Save those 128 measurements.

4. Looking at all the faces we’ve measured in the past, see which person has the closest measurements to our face’s measurements. That’s our match!

You can learn in-depth about this here.

4.3 Sending Alert

from boltiot import Bolt
import conf, time #import device api key and ids along with telegram details
import telegram

from PIL import Image #for the frame(numpy array) to be coverted to PIL image 

from telegram.ext import Updater
from telegram.ext import CommandHandler, CallbackQueryHandler
from telegram import InlineKeyboardButton, InlineKeyboardMarkup 

Here conf is conf.py that we created before. Since the alert function accepts the frame (numpy array) as a parameter, we have to convert it to a PIL image object to be sent to phone over Telegram.

 def alert(frame):

	#we convert numpy array frame to an RGB image and save as my.png
        #because only images are sent via the telegram bot
	img = Image.fromarray(frame, 'RGB')
	img.save('my.png')
        #configure telegram chat bot
	bot = telegram.Bot(token=conf.TELEGRAM_BOT_ID) 
	
	#configuring and switching on the buzzer
        #and switching off after a certain time
	mybolt = Bolt(conf.API_KEY, conf.DEVICE_ID)
	response = mybolt.digitalWrite('0', 'HIGH')
	
	#We send the captured photo in binary mode of the intruder 
        #along with a caption to telegram
	try:
		bot.send_photo(chat_id=conf.TELEGRAM_CHAT_ID, photo=open('my.png', 'rb'), caption="Intruder Alert!")
 
	except Exception as e: #raise exception if not sent
		print("An error occurred in sending the message via Telegram")
		print(e)
	#switch off the buzzer
	response = mybolt.digitalWrite('0', 'LOW')
#end alert

5. Conclusion

Face recognition technology has advanced manifolds since its inception and a lot of research is currently going on to further improve it. The project I created aims to combine already existing technologies and prevent fake/spoofed faces from gaining access.