Multi-Modal Biometric Attendance System

High-Level Overview

• Face & Speech-based Attendance Registration Gadget with OpenVino
• Can be used in big companies, flats & institutions or even class attendance.
• Better accuracy than Face Recognition using HAARCASCADE, LBPH, HOG and SVM.
• 3 Steps:

a) Detect Face on Edge (Raspberry Pi + Pi Cam + Movidius)

b) Publish Identity to server (Use Flask or MQTT)

c) Persist identity and attributes (Backend DB & Frontend)

• Alert Mechanism to Security, if guest is unidentified (SMS or MMS)
• Multimodal Biometric: If vision fails, identify the person with sound bits (Kaldi)
• Activity Recognition and Anti-Spoofing methods (2D & 3D).

Project Criteria

Salient Features

• Anti-spoofing methods:

Eye blink detection (2D): Detect Eye Aspect Ratio (if EAR < δ for 250ms then natural blinking)

Active Flash (3D): detect spoofing using light reflections on a face

• Two Types of Scaling:

Multiple Cams: Network wont be jammed as inference is on the edge.

Too many people: Parallel vector ops. Async Communication using MQTT (Not GET/ POST)

• Alert Mechanism: Footage of Unidentified Persons notified to security via FFmpeg (MMS)
• Protected Areas: Give personalized audio response, when an employee is not allowed.
• Activity Heatmap based on people activity inside company (Analytical Info for HR)
• Person Location Map: People can be searched inside big MNCs based on 'Last Seen Info'.

Project Showcase Criteria

Innovation & Creativity

Practical and ideal use-case for edge deployment which can replace or assist humans

Bought Raspberry Pi 4 & Intel Movidius NCS 2 out of interest to do this project

Installed OpenVino in local Ubuntu & Raspian in Raspberry Pi 4 from scratch including OpenCV

Relevance & Potential

Attendance is ubiquitous. Customers are available who need vision-based atttendance management

Plan is to register this as startup, once the system matures.

Huge Market Potential: big companies, MNCs, institutions, school & college classrooms/ exams etc.

Model Accuracy & Demonstration of the Course Material

Out of 15+ pre-trained models in OV Model Zoo, Face Recognition models has highest accuracy

Covers leveraging pre-trained Models, MO, IE, and focus was on Edge Deployment (Lessons 2~5)

Better accuracy than Face Recognition using HAARCASCADE, LBPH, HOG and SVM.

Speed, Robustness & Fairness

Handle boundary cases with alert mechanism & anti-spoofing methods for robustness.

Going to do multithreading on multiple NCS to push speed to 20+ FPS.

Parallel vector operations for face comparison using np.linalg.norm function and matrix-vector substraction for scalability.

Modules to be completed...

• Need to build communication interface with MQTT
• Need to build DB for save and retrieve
• Web-interface based on database yet to build.
• Twilio module is to be done.
• Anti-spoofing Modules
• Biometric Fallback mechanism (Speech Module)
• Dynamic cut-off setting based on FAR & FRR
• Handling “Unknown” with MMS trigger.
• Activity Heatmap & Person Map
• Deploy or re-implement in alternate boards like BeagleBoard AI, Google Coral, Jetson Nano, Ambaralla CV25, Kendryte’s KD233 or Grove AI HAT with RPi/ Nano.

Notification of Identified Persons: