Created December 20, 2024

RPi Zero W Person Sensor Testbed

A previously unfinished project to test the Useful Sensors Person Sensor upgraded to use the latest available hardware components.

Things used in this project

Hardware components

Raspberry Pi Zero Wireless

Raspberry Pi Zero 2 W

Raspberry Pi Camera Module

Raspberry PI AI Camera

Useful Sensors Person Sensor

Software apps and online services

KiCad

Seeed Studio Fusion PCB Fabrication

Node-RED

MQTT

Raspberry Pi Raspbian

Hand tools and fabrication machines

Anycubic Kobra 2 3D Printer

Story

Introduction

When the Useful Sensors Person Sensor came out a couple of years ago I was impressed by its tiny size and low cost. I was also intrigued by the Sensor 2.0 initiative for data privacy. The Person Sensor is performing face detection and recognition but biometric information is not available from the device. It is only used for detection on the sensor. The sensor only provides face bounding box location within the image frame and the detection confidence. It can also match a stored image that is also not accessible and provide additional information as to whether the pose of the head is facing the sensor (eye direction is not detected).

When testing use cases for image sensors I find it useful to be able to view the image that the sensor sees and correlate that to the detection metadata stream. Since the image data is not available from the sensor I thought that I would build a simple testbed using a RPi Zero W and RPi Camera mounted adjacent to the Person Sensor to capture the approximate image that it was seeing.

That was the genesis of the project recovered from my junk drawer. Unfortunately, it was one of those projects that started but never finished because other higher value projects came along.

Here are the pieces that came out of the "junk drawer" and an assembly showing how far I got.

Parts:

RPi Zero W V1.1 with acrylic case
RPi Camera V1.3
Person Sensor V1.0
3D printed tripod mount for the assembly

Partially completed assembly (Person Sensor was not mounted). It was going to be mounted directly above the camera.

In the past two years, the Person Sensor has gone through a couple of revisions and the RPi Zero now has a much more capable 2 W version. I decided that I'd like to rebuild the project using the newer components and maybe throw in an RPi AI Camera for good measure. In the spirit of building from junk drawer components, I'll also build a variant using the original parts.

PCB Design

PCBs have gotten really inexpensive to design and fabricate and with quick fabrication times, so I thought that I would create a simple dedicated micro HAT. The HAT is very simple and just used for mechanical structure and to interface the I2C and power to the Person Sensor (the newer Person Sensors have header pins, but to use the original one I'll need a QWIIC adapter cable).

I did a quick and dirty PCB design in KiCad - 2 layers with only 3.3V, Gnd,,and I2C connections to the GPIO header. I had it fabricated by Seeed Studio Fusion. It cost $4.90 for the 10 board minimum and $13 shipping for a 10 day turnaround. $1.80 a board is much cheaper than anything I could have done myself (of course, I doubt that I'll ever use 10 boards). If I had more time I might have tried a more generic PCB - maybe UART, SPI, and I2C and QWIIC or Grove connectors.

PCB Front and Back

I then 3D printed a tripod adapter to hold the RPi Zero/uHat assembly in a vertical orientation.

Final Project Assembly

The finished assembly using the newest version of the components

RPi Zero 2 W
RPi AI Camera
Person Sensor V2.0

A short video to show the operation of the testbed. The screen has 3 sections.

Upper left is the RPi AI Camera output running mobilenet_ssd
Lower left is a terminal showing the I2C output from the Person Sensor
Right side is a webcam image of the testbed assembly

The Person Sensor has a green LED that lights to indicate that it has detected a face. It also outputs the following data via I2C:

Number of faces detected
Confidence of detected face
Bounding box location of detected face
Recognition confidence if the face id has been registered
Face ID detected
Whether the the face is looking at the sensor -"is_facing"

Example of the data printed to the terminal:

A few observations from the video. First, the Person Sensor is trained to detect a face rather than a person. The Person Sensor has a high confidence detection as long as I'm looking straight into the sensor. If I look up or to the left or right the sensor will go through a transition where it determines that I am not "facing" it and then will no longer detect my face. It does better when I am facing down but does drop out intermittently. It also will lose detection if I obscure my face using my hand. The RPi AI Camera using a dataset trained on "person" data always detects me as a person even when my face is obscured or not facing it.

MQTT Interface to Node-Red

I am using MQTT to send the detection information to a local MQTT server running on an RPi4. I'll use Node-Red to display the information on a dashboard.

The RPi Zero formats the Person Sensor data into a JSON object that is sent via MQTT message to the server.

Output from the MQTT input face_detection node:

MQTT message payload

The overall Node-Red flow:

Screen capture of the dashboard display:

Video showing showing the bounding box info updating:

The blue rectangle shows the position of the bounding box in the Person Sensor frame. Size of the bbox is a proxy for proximity.

Project Variants

Since I had extra PCBs I built two variants. Both are using non-AI RPi cameras so basically just act like a view finder for the Person Sensor.

The first variant uses the original version of the Person Sensor. This version did not break out the I2C or power to header pins and did not have mounting holes. I 3D printed a holder for the sensor and added a QWIIC pigtail cable to interface to the PCB. This sensor version also had the "up" orientation with the QWIIC connector on top. The newest sensor has the orientation with the connector on the bottom so I have flipped the PCB 180 degrees when attaching to the tripod mount. This variant uses the RPi V1.3 camera and the RPi Zero W..

Screencap of Person Sensor Testbed running with the original junk drawer parts:

The second variant uses the new Person Sensor and the RPi V2.1 camera and the RPi Zero W.

I'm using longer camera cables on the variants because I don't have any more of the short ones. I only use a heatsink on the processor with the Zero 2 W and not the Zero W variants. The I2C bus used on the Zero W is /dev/i2c-0 and on the Zero 2 W it is /dev/i2c-1 so that also requires a software change.

Custom parts and enclosures

Sketchfab still processing.

Schematics

Code

# Example of accessing the Person Sensor from Useful Sensors on a Pi Zero using
# Python. See https://usfl.ink/ps_dev for the full developer guide.

import io
import fcntl
import struct
import json
import random
import time

import paho.mqtt.client as mqtt

def on_connect(client, userdata, flags, rc):

    print("Connected with result code " + str(rc))

client = mqtt.Client()

client.on_connect = on_connect

client.connect("10.0.0.234", 1883)

# The person sensor has the I2C ID of hex 62, or decimal 98.
PERSON_SENSOR_I2C_ADDRESS = 0x62

# We will be reading raw bytes over I2C, and we'll need to decode them into
# data structures. These strings define the format used for the decoding, and
# are derived from the layouts defined in the developer guide.
PERSON_SENSOR_I2C_HEADER_FORMAT = "BBH"
PERSON_SENSOR_I2C_HEADER_BYTE_COUNT = struct.calcsize(
    PERSON_SENSOR_I2C_HEADER_FORMAT)

PERSON_SENSOR_FACE_FORMAT = "BBBBBBbB"
PERSON_SENSOR_FACE_BYTE_COUNT = struct.calcsize(PERSON_SENSOR_FACE_FORMAT)

PERSON_SENSOR_FACE_MAX = 4
PERSON_SENSOR_RESULT_FORMAT = PERSON_SENSOR_I2C_HEADER_FORMAT + \
    "B" + PERSON_SENSOR_FACE_FORMAT * PERSON_SENSOR_FACE_MAX + "H"
PERSON_SENSOR_RESULT_BYTE_COUNT = struct.calcsize(PERSON_SENSOR_RESULT_FORMAT)

# I2C channel 1 is connected to the GPIO pins
I2C_CHANNEL = 1
I2C_PERIPHERAL = 0x703

# How long to pause between sensor polls.
PERSON_SENSOR_DELAY = 0.2
#PERSON_SENSOR_DELAY = 1.0

i2c_handle = io.open("/dev/i2c-" + str(I2C_CHANNEL), "rb", buffering=0)
fcntl.ioctl(i2c_handle, I2C_PERIPHERAL, PERSON_SENSOR_I2C_ADDRESS)

while True:
    try:
        read_bytes = i2c_handle.read(PERSON_SENSOR_RESULT_BYTE_COUNT)
    except OSError as error:
        print("No person sensor data found")
        print(error)
        time.sleep(PERSON_SENSOR_DELAY)
        continue
    offset = 0
    (pad1, pad2, payload_bytes) = struct.unpack_from(
        PERSON_SENSOR_I2C_HEADER_FORMAT, read_bytes, offset)
    offset = offset + PERSON_SENSOR_I2C_HEADER_BYTE_COUNT

    (num_faces) = struct.unpack_from("B", read_bytes, offset)
    num_faces = int(num_faces[0])
    offset = offset + 1

    faces = []
    for i in range(num_faces):
        (box_confidence, box_left, box_top, box_right, box_bottom, id_confidence, id,
         is_facing) = struct.unpack_from(PERSON_SENSOR_FACE_FORMAT, read_bytes, offset)
        offset = offset + PERSON_SENSOR_FACE_BYTE_COUNT
        face = {
            "box_confidence": box_confidence,
            "box_left": box_left,
            "box_top": box_top,
            "box_right": box_right,
            "box_bottom": box_bottom,
            "id_confidence": id_confidence,
            "id": id,
            "is_facing": is_facing,
        }
        faces.append(face)
    checksum = struct.unpack_from("H", read_bytes, offset)
    print(num_faces, faces)
    
    # Convert to JSON string
    json_data = json.dumps(faces, indent=4)

    # Print the JSON data
    #print(json_data)
    
    if (len(faces) > 0):
        print(json_data)
        client.publish("person_sensor/face_detection", json_data)
    
    #client.loop_forever()
    
    time.sleep(PERSON_SENSOR_DELAY)

[
    {
        "id": "0b9e029045fa445a",
        "type": "tab",
        "label": "Person Sensor Testbed",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "0864e53c4fee9221",
        "type": "mqtt in",
        "z": "0b9e029045fa445a",
        "name": "face_detection",
        "topic": "person_sensor/face_detection",
        "qos": "2",
        "datatype": "json",
        "broker": "98ad732.a6d0b9",
        "nl": false,
        "rap": true,
        "rh": 0,
        "inputs": 0,
        "x": 260,
        "y": 640,
        "wires": [
            [
                "69361fc3fd8d49f8",
                "76c29d90e7d12f17",
                "099a2cd5faa83bb8",
                "373f30b1a8602e40"
            ]
        ]
    },
    {
        "id": "69361fc3fd8d49f8",
        "type": "debug",
        "z": "0b9e029045fa445a",
        "name": "face_detection",
        "active": false,
        "tosidebar": true,
        "console": false,
        "tostatus": true,
        "complete": "true",
        "targetType": "full",
        "statusVal": "payload",
        "statusType": "auto",
        "x": 740,
        "y": 460,
        "wires": []
    },
    {
        "id": "76c29d90e7d12f17",
        "type": "ui_text",
        "z": "0b9e029045fa445a",
        "group": "7289634b3976b97b",
        "order": 1,
        "width": 0,
        "height": 0,
        "name": "",
        "label": "Box Confidence",
        "format": "{{msg.payload[0].box_confidence}}",
        "layout": "row-spread",
        "x": 740,
        "y": 560,
        "wires": []
    },
    {
        "id": "099a2cd5faa83bb8",
        "type": "ui_text",
        "z": "0b9e029045fa445a",
        "group": "7289634b3976b97b",
        "order": 2,
        "width": 0,
        "height": 0,
        "name": "",
        "label": "Is_Facing",
        "format": "{{msg.payload[0].is_facing}}",
        "layout": "row-spread",
        "x": 720,
        "y": 600,
        "wires": []
    },
    {
        "id": "43cc2fa56109df44",
        "type": "debug",
        "z": "0b9e029045fa445a",
        "name": "face location",
        "active": false,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 950,
        "y": 800,
        "wires": []
    },
    {
        "id": "7f4fd912b756b6a1",
        "type": "ui_text",
        "z": "0b9e029045fa445a",
        "group": "7289634b3976b97b",
        "order": 3,
        "width": 0,
        "height": 0,
        "name": "",
        "label": "Box_X",
        "format": "{{msg.x}}",
        "layout": "row-spread",
        "x": 710,
        "y": 640,
        "wires": []
    },
    {
        "id": "117d9e832a96a7b6",
        "type": "ui_text",
        "z": "0b9e029045fa445a",
        "group": "7289634b3976b97b",
        "order": 4,
        "width": 0,
        "height": 0,
        "name": "",
        "label": "Box_Y",
        "format": "{{msg.y}}",
        "layout": "row-spread",
        "x": 710,
        "y": 680,
        "wires": []
    },
    {
        "id": "e66e288705d3ff91",
        "type": "ui_text",
        "z": "0b9e029045fa445a",
        "group": "7289634b3976b97b",
        "order": 5,
        "width": 0,
        "height": 0,
        "name": "",
        "label": "Box_W",
        "format": "{{msg.w}}",
        "layout": "row-spread",
        "x": 720,
        "y": 720,
        "wires": []
    },
    {
        "id": "bbf5aca6db7eeb24",
        "type": "ui_text",
        "z": "0b9e029045fa445a",
        "group": "7289634b3976b97b",
        "order": 6,
        "width": 0,
        "height": 0,
        "name": "",
        "label": "Box_H",
        "format": "{{msg.h}}",
        "layout": "row-spread",
        "x": 710,
        "y": 760,
        "wires": []
    },
    {
        "id": "373f30b1a8602e40",
        "type": "function",
        "z": "0b9e029045fa445a",
        "name": "bbox",
        "func": "var x = msg.payload[0].box_left;\nmsg.x = x;\nvar y = msg.payload[0].box_top;\nmsg.y = y;\nvar w = msg.payload[0].box_right - msg.payload[0].box_left;\nmsg.w = w;\nvar h = msg.payload[0].box_bottom - msg.payload[0].box_top;\nmsg.h = h;\nreturn msg;",
        "outputs": 1,
        "timeout": 0,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 490,
        "y": 640,
        "wires": [
            [
                "7f4fd912b756b6a1",
                "117d9e832a96a7b6",
                "e66e288705d3ff91",
                "bbf5aca6db7eeb24",
                "29541855f532d124",
                "4f15439d93735d11"
            ]
        ]
    },
    {
        "id": "29541855f532d124",
        "type": "debug",
        "z": "0b9e029045fa445a",
        "name": "bbox",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 710,
        "y": 900,
        "wires": []
    },
    {
        "id": "4f15439d93735d11",
        "type": "ui_template",
        "z": "0b9e029045fa445a",
        "group": "7289634b3976b97b",
        "name": "Face Location",
        "order": 7,
        "width": 13,
        "height": 13,
        "format": "<canvas id=\"myCanvas\" width=\"256\" height=\"256\" style=\"border:1px solid\"></canvas>\n\n<script>\n\n\n(function(scope) {\n    // $watch fires each time the node is triggered in the flow\n    scope.$watch('msg', function(msg) {\t\n\n    const canvas = document.getElementById(\"myCanvas\");\n    const ctx = canvas.getContext(\"2d\");\n    ctx.fillStyle = \"blue\"; // Set fill color\n\n  \tctx.clearRect(0, 0, canvas.width, canvas.height);\n\n    ctx.fillStyle = \"blue\";\n\n  \tctx.fillRect(msg.x, msg.y, msg.w, msg.h); // Draw rectangle\n\n    });\n\n})(scope);  \n</script>",
        "storeOutMessages": false,
        "fwdInMessages": true,
        "resendOnRefresh": false,
        "templateScope": "local",
        "x": 740,
        "y": 800,
        "wires": [
            [
                "43cc2fa56109df44"
            ]
        ]
    },
    {
        "id": "98ad732.a6d0b9",
        "type": "mqtt-broker",
        "name": "",
        "broker": "10.0.0.234",
        "port": "1883",
        "clientid": "",
        "autoConnect": true,
        "usetls": false,
        "compatmode": false,
        "protocolVersion": "4",
        "keepalive": "60",
        "cleansession": true,
        "autoUnsubscribe": true,
        "birthTopic": "",
        "birthQos": "0",
        "birthPayload": "",
        "birthMsg": {},
        "closeTopic": "",
        "closeQos": "0",
        "closePayload": "",
        "closeMsg": {},
        "willTopic": "",
        "willQos": "0",
        "willPayload": "",
        "willMsg": {},
        "userProps": "",
        "sessionExpiry": ""
    },
    {
        "id": "7289634b3976b97b",
        "type": "ui_group",
        "name": "Person Sensor",
        "tab": "2d394dc643d3b04a",
        "order": 1,
        "disp": true,
        "width": 13,
        "collapse": false
    },
    {
        "id": "2d394dc643d3b04a",
        "type": "ui_tab",
        "name": "Person Sensor",
        "icon": "dashboard",
        "disabled": false,
        "hidden": false
    }
]

RPi Zero W Person Sensor Testbed

Things used in this project

Hardware components

Software apps and online services

Hand tools and fabrication machines

Story

Introduction

PCB Design

Final Project Assembly

MQTT Interface to Node-Red

Project Variants

Custom parts and enclosures

RPi_Zero_2W_tripod_Mount

Schematics

Person_Sensor_uHat schematic

Code

person_sensor_mqtt_code.py

person_sensor_testbed_flow.json

Credits

Ralph Yamamoto

Comments

Embed the widget on your own site

RPi Zero W Person Sensor Testbed

RPi Zero W Person Sensor Testbed

Things used in this project

Hardware components

Software apps and online services

Hand tools and fabrication machines

Story

Introduction

PCB Design

Final Project Assembly

MQTT Interface to Node-Red

Project Variants

Custom parts and enclosures

RPi_Zero_2W_tripod_Mount

Schematics

Person_Sensor_uHat schematic

Code

person_sensor_mqtt_code.py

person_sensor_testbed_flow.json

Credits

Ralph Yamamoto

Comments