Foolproof Temperature Checker Amid COVID 19 Pandemic

1.0 PROBLEM STATEMENT

Wearing facemask in public is already a mandatory order for some countries following COVID 19 pandemic and one of the standard operating procedures implemented by local government here is to have the body temperature checked prior entering a building or any premise, access will be denied if the body temperature exceed 37.5degC. Currently, human intervention is required to ensure compliance.

2.0 PROPOSAL

Many places have the temperature checker which consists of a conventional IR thermometer with display or buzzer mounted to a tripod setup at the building door step or entrance. What the proposal emphasizes is, to create a foolproof temperature checker which will use low power vision sensor with embedded controller, to run machine learning algorithm for identifying if the person is wearing facemask before checking temperature. Besides, the checker will also connect with AWS IoT Core to confirm if to enable/disable checking, depends on the number of people occupying the premise if exceed the threshold.

3.0 HOW TO

Schematic diagrams is plan as below. Added on 2021107.

3.1 Image Classification at Camera

OpenMV H7 plus is used for running image classification onboard. First off, dataset images which are classified as MarkYes, MaskNo & Hand are collected in the OpenMV IDE which will then upload to EDGE IMPULSE for neural network training. Take note that the images should be sized/resized to 96x96 since we are running machine learning on microcontroller. Once all dataset are ready for feature extraction we can configure the pipeline parameter, such as training cycle, learning rate, transfer learning model (MobileNetV2 model is used in this project). etc. Training result is remarkable as shown below.

Firmware file generated by EDGE IMPULSE (*.tflite) should be copied to OpenMV camera together with the label text file. Once done, running image classification is easy by simply calling out the built-in tensorflow module by OpenMV. Mask detected is coded to run consecutive 5 cycles to ensure no misdetection then proceed to read temperature. UART command 1 will be sent from OpenMV to Core2 via port C at this point to trigger led blue blinking. Temperature reading will be averaged out across interval of roughly 3sec to give stable value. UART command 2 will be sent again at this point to indicate completion of temperature reading and also send the value to core2. Core2 will evaluate if the temperature is within range and trigger action accordingly.

3.2 Temperature Sensing by IR Sensor

MLX90614 is a cheap infrared sensor that is readily available at local store, but this sensor is not recommended for those who needs accuracy. I find that the detection range is short plus the reading is not consistent. Anyway, this sensor runs on I2C hence a prototype board was soldered to connect it to the OpenMV camera as shown in image below. The I2C address is fixed at 0x5A and you can read to the memory register for object and environment temperature.

3.3 Grove Mini PIR Sensor

This sensor is connected to the port B of Core2. Pay attention to the REL pin of mini PIR sensor is actually leading to GPIO38 of Core2 if using the standard Grove 4-wire cable, so need to swap the wire GPIO26 manually. The purpose of this sensor is meant to detect human passing by the exit of a premise, to trigger the AWS IoT Event to decrease the counting of people in premise. However, the sensor turns faulty just after few days of which led to change of plan during the video demonstration.

3.4 Core2 for AWS IoT Edukit

Significant amount of time was spent going through the learning curve for, not only for the coding with ESPIDF framework in PlatformIO but also FreeRTOS as well as LVGL. Fortunately the tutorial https://edukit.workshop.aws/en/ is well designed and it's really helpful to get things started. The code for this project is based on the smart thermostat example, adding:

• UART communication with OpenMV camera
• Modified MQTT topic with AWS IoT Core
• Corresponding visualization changes

The design workflow for this project is illustrated below:

The Core2 visualization is updated later on 20211105 to following:

• Threshold setting in AWS IoT Event indicate how many persons are allowed in the premise.
• Counter value in AWS IoT Event for occupancy tracking in the premise.
• Message to inform user when taking measurement, temperature OK/NG, when premise is full.

3.5 AWS IoT Core

As taught in the tutorial, a simple rule query is setup in AWS IoT Core to forward the MQTT topic as input for AWS IoT Event, where the detector model will use as input for the state/event trigger. The detector model is illustrated in below and the exported json file can be found in the code. One thing to take note is my model is using serial evaluation which will evaluate the logic condition follow sequence. P/S: There is one thing I still wish to achieve is how to publish a variable counter from AWS IoT Event back to the device, but this is not a show stopper for my project anyway. --> This is updated on 20211105.

Two mistakes that were holding up the publish of variable from AWS IoT Event back to the device are:

• The syntax for publishing variable in json expression in AWS IoT Event needs to start with ```${}``` and a de-refencing $ in it. Refer to doc here, https://docs.aws.amazon.com/iotevents/latest/developerguide/expression-usage.html. Example lines in my detector script,

"contentExpression": "'{ \"state\": { \"desired\": { \"testDesired\": true, \"thres_fr_aws\": ${$variable.countThreshold}, \"count_fr_aws\": ${$variable.testReported_count} } } }'"

• The variable published by AWS IoT Core to device needs function below to register the delta of shadow update. (Of course, the variable needs to be declared in advance as jsonstruc_t together with its members). Example lines below are what were missing before the update.

rc = aws_iot_shadow_register_delta(&iotCoreClient, &awsThres);
rc = aws_iot_shadow_register_delta(&iotCoreClient, &awsCount);

3.6 3D-printed Case

A case being the last piece of the project to hold everything together.

4.0 OUTCOME

The final product is considered mission more-than-half-accomplished as the facemask checking part is working delightfully, with one exception on the mini PIR sensor turns faulty after just few days of testing (had confirmed it can take 5V) which would have been making the occupancy control really close to production-like. Here is a video demonstration below, demonstrating the features of foolproof temperature checker:

• NO MASK NO CHECK
• TOO MANY NO CHECK TOO (in simulation mode)

Video below is updated later, one month after the contest closure, with the access control feature finally is working between Core2 and the AWS IoT Core & Event.

Last word to conclude my project, I think this contest is really exciting and fruitful. The hardware is top-notch quality packed with so many features, AWS IoT is a even much bigger arena to explore, I can only blame myself for not having enough time to try that out. For sure I will continue come back with some more projects in future.

Update:

20211007 - Rectified the mistake of mixing up between port B & port C & added null terminator to the uart string in openmv py script.

20211105 - Update with part 2 demo for access control with AWS IoT Core & Event, with GitHub repository.