Installing the Required Libraries

Created December 29, 2019

Classroom Attendance Tracker

Easily track attendance in large classroom settings

AdvancedFull instructions provided15 hours70

Things used in this project

Hardware components

NVIDIA Jetson Nano Developer Kit

Flash Memory Card, MicroSD Card

Webcam, Logitech® HD Pro

Power Supply 5V 4A

Story

In large lecture halls in many universities, it is difficult if not impossible to take attendance, leaving professors to either resort to using annoying online forms or clickers which can easily be manipulated or to not taking attendance at all, which results in poor attendance and loss of learning. A device that can perform facial recognition is perfect for solving this problem.

With the NVIDIA Jetson Nano Developer Kit, this project can be made to have a low profile on the ceiling or doorway of any lecture hall, scanning faces and taking attendance as lecture occurs without causing any disruption to class. This is different from existing solutions because the current attendance systems all require input from either the professor or students. In addition, there are very few solutions that are accurate to make sure nobody is gaming the system. It is extremely useful because class attendance is necessary for learning, but taking attendance takes away time for learning, so this system allows learning to happen to its fullest potential. The Jetson Nano Developer Kit is powerful enough to do all this processing on the board itself, requiring no internet connection. It also is compact enough to not be a distraction. The main features of this are to take attendance and learn different appearances so teachers and students do not need to take class time to take attendance.

To build this project, I first set up the Jetson Nano using Nvidia's provided guide here: https://developer.nvidia.com/embedded/learn/get-started-jetson-nano-devkit. This uses the NVIDIA JetPack SDK to include CUDA libraries for hardware-accelerated vision processing.

Then, I plugged in a keyboard, mouse, monitor, and UVC-compatible USB webcam to be able to use the Jetson Nano.

You may also use a camera that plugs into the MIPI CSI camera connector. The code (in this case) is the same.

When you plug in the Micro-USB power cable to the Jetson Nano, it will automatically boot up, and you will go through the standard Ubuntu new user setup process.

Installing the Required Libraries

Python 3.6 and OpenCV are already installed with the Jetson Nano version of Ubuntu. However, to run vision processing we need a few more.

Type the following commands into your terminal:

sudo apt-get update

sudo apt-get install python3-pip cmake libopenblas-dev liblapack-dev libjpeg-dev

-------------------
# These next two allow the Jetson to use storage space as RAM.
git clone https://github.com/JetsonHacksNano/installSwapfile

./installSwapfile/installSwapfile.sh

-------------------

pip3 install numpy

-------------------

wget http://dlib.net/files/dlib-19.17.tar.bz2 

tar jxvf dlib-19.17.tar.bz2

cd dlib-19.17

gedit dlib/cuda/cudnn_dlibapi.cpp

# scroll to line 854 and comment out the line. It should now read: //forward_algo = forward_best_algo;
# save the file, then return to your terminal and type the following. It will take about 30 minutes.
sudo python3 setup.py install

# now we need to install a custom library for facial recognition that makes our code much simpler!
sudo pip3 install face_recognition

After installing all of these libraries, you are ready to use your Jetson Nano for face recognition!

Writing the Code

See my code file below called main.py. It is commented to allow you to better understand the code.

In short, the code file uses OpenCV to capture video from the webcam, then uses linear algebra techniques to match the face it sees to one in the list of known faces. Then, once it matches a face, it will add the name and email to a text file.

The next file, called helper.py, takes the names and emails from the text file and sends an email using SMTP to each email in the file. This way, students will receive an email confirmation that they have been detected walking into the door.

After setup, the system can be run with only power, the webcam, and Ethernet plugged in. The HDMI, mouse, and keyboard can be unplugged. An optional USB wifi dongle can be added in place of the Ethernet cable.

If internet is not accessible, the list of students is saved locally onto the Jetson Nano's SD card and can be used to track attendance without internet access.

Next Steps

In the future, I hope to implement a deep learning model into this software, so that the software can detect faces more accurately, if a student has changed their appearance. This way there will be fewer false positives and negatives and the system will be more accurate. As is, however, it definitely will achieve its goal of saving professors and students time.

Note: Thanks to @ageitgey on Medium and Github for providing a guide on which I based my project. Used under the MIT License.

#!/usr/bin/python3  
import smtplib

sender_mail = 'sender@gmail.com' # attendance email
pwd = 'password' # attendance email account password
smtpObj = smtplib.SMTP('gmail.com',587)  
smtpObj.login(sender_mail,password)  
f = open("attendance.txt","r")
lines = f.readlines()
for line in lines:
  student_name = line[:line.find(",")] # file is formatted so that name and email are separated with a comma.
  student_email = line[(line.find(",") + 1):]
  receivers_mail = [student_email]  
  message = """Dear %s,
  Your attendance has been recorded. 
  """%(student_name)  
  try:   
     smtpObj.sendmail(sender_mail, receivers_mail, message)  
     print("Successfully sent email")  
  except Exception:  
     print("Error: unable to send email")  
f.close()

import face_recognition
import cv2
import numpy as np
import os

f = open("attendance.txt","w+")

video_capture = cv2.VideoCapture(0) # Captures video from the default camera. 

path = '/' # Uses this path to look for student image files (files stored in JPG format) File name is as follows: "First Last, Email.jpg"

known_face_encodings = []
known_face_names = []
for root, directories, files in os.walk(path):
    for file in files:
        if '.jpg' in file:
            current_image = face_recognition.load_image_file(os.path.join(root, file))
            current_face_encoding = face_recognition.face_encodings(current_image)[0]
            known_face_encodings.append(current_face_encoding)
            known_face_names.append(file[:-4]) # Adds names and emails into list without the .jpg part of the filename

# Initialize variables
face_locations = []
face_encodings = []
face_names = []
process_this_frame = True

while True:
    # Grab a single frame of video
    ret, frame = video_capture.read()

    # Resize frame of video to 1/2 size for faster face recognition processing
    small_frame = cv2.resize(frame, (0, 0), fx=0.5, fy=0.5)

    # Convert the image from BGR color (which OpenCV uses) to RGB color (which the face_recognition library uses)
    rgb_small_frame = small_frame[:, :, ::-1]

    # Only process every other frame of video to save time
    if process_this_frame:
        # Find all the faces and face encodings in the current frame of video
        face_locations = face_recognition.face_locations(rgb_small_frame)
        face_encodings = face_recognition.face_encodings(rgb_small_frame, face_locations)

        face_names = []
        for face_encoding in face_encodings:
            # See if the face is a match for the known face(s)
            matches = face_recognition.compare_faces(known_face_encodings, face_encoding)
            name = "Unknown"

            # Use the known face with the smallest distance to the new face
            face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)
            best_match_index = np.argmin(face_distances)
            if matches[best_match_index]:
                name = known_face_names[best_match_index]
                f.write(name) # adds name to file to be processed by helper.py

            face_names.append(name)

    process_this_frame = not process_this_frame # Process every other frame

    # Display the results
    for (top, right, bottom, left), name in zip(face_locations, face_names):
        # Scale back up face locations since the frame we detected in was scaled to 1/2 size
        top *= 2
        right *= 2
        bottom *= 2
        left *= 2

        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)

        # Draw a label with a name below the face
        cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255), cv2.FILLED)
        font = cv2.FONT_HERSHEY_DUPLEX
        cv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)

    # Display the resulting image
    cv2.imshow('Video', frame)

    # Hit 'q' on the keyboard to quit!
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release handle to the webcam
video_capture.release()
cv2.destroyAllWindows()
f.close()

Credits

Jerry Zhang

11 projects • 15 followers

Embed the widget on your own site

Classroom Attendance Tracker

Classroom Attendance Tracker

Things used in this project

Hardware components

Story

Installing the Required Libraries

Code

helper.py

main.py

Credits

Jerry Zhang

Comments

Related channels and tags