virtual joystick with face movement

1.Project Planning

Machine learning and deep learning have brought about a revolution in artificial intelligence, and more and more visual information can be recognized by machines, and face recognition technology is a typical representative of it. We use the face recognition module to locate the movement of the face, and can use the up, down, left and right movements of the face to control the game character, so as to achieve the effect of a virtual gamepad. This will help people with disabilities to input the game's movements, so that everyone can enjoy the game.

2.Part Sourcing

In order to achieve this goal, we use the Blue Swan board as the main control board, the Useful Sensors Person Sensor as the face recognition module, and Arduino as the development platform for rapid development:

1） Blues Swan

2） Useful Sensors Person Sensor

3） Qwiic wires

3. Building

The main body of the circuit is the Blues Swan.The development board is connected to the Person Sensor through the I2C interface. We can see serial print information through virtual com port.

The entire circuit is shown in the figure.

4.Coding

We need to communicate through the I2C with person sensor modules, collect the position of the face from it, and output the change of the face position to the serial port for printing by analyzing the position change of the face, or control the output through the GPIO.

#include <Wire.h>
#include "person_sensor.h"
// How long to wait between reading the sensor. The sensor can be read as
// frequently as you like, but the results only change at about 5FPS, so
// waiting for 200ms is reasonable.
const int32_t SAMPLE_DELAY_MS = 200;
void setup() {
    // You need to make sure you call Wire.begin() in setup, or the I2C access
    // below will fail.
    Wire.begin();
    Serial.begin(9600);
}

void loop() {
    person_sensor_results_t results = {};
    // Perform a read action on the I2C address of the sensor to get the
    // current face information detected.
    if (!person_sensor_read(&results)) {
        Serial.println("No person sensor results found on the i2c bus");
        delay(SAMPLE_DELAY_MS);
        return;
    }
    Serial.println("********");
    Serial.print(results.num_faces);
    Serial.println(" faces found");
    for (int i = 0; i < results.num_faces; ++i) {
        const person_sensor_face_t* face = &results.faces[i];
        Serial.print("Face #");
        Serial.print(i);
        Serial.print(": ");
        Serial.print(face->box_confidence);
        Serial.print(" confidence, (");
        Serial.print(face->box_left);
        Serial.print(", ");
        Serial.print(face->box_top);
        Serial.print("), (");
        Serial.print(face->box_right);
        Serial.print(", ");
        Serial.print(face->box_bottom);
        Serial.print("), ");
        if (face->is_facing) {
            Serial.println("facing");
        } else {
            Serial.println("not facing");
        }
    }
    delay(SAMPLE_DELAY_MS);
}