Voice controlled door access with Picovoice and PSoc 6

.

Overview

.

Door access normally is controlled by passcode with keypad or key card. In this project I am using voice to spell the passcode for door access. The flow is simple. The user will say the name and the passcode and the edge device will capture these info (name and passcode) using Picovoice app running on a PSoC 6 device. If the name and passcode are matched, they will be shown on the OLED display together with a message of "Access allowed"..

.

Picovoice - Door Access App

.

In my earlier post, I described how to build and program a Picovoice app on to a PSoC 6 device. Opened Eclipse IDE for ModusToolbox and created a new Picovoice app called picovoice_app.

.

Custom Wake Word

.

Next I created and trained the default wake word to "hey melissa" at the Picovoice console:

.

.

Downloaded the model:

.

.

Unzipped the model and copied the KEYWORD_ARRAY from the pv_porcupine_params.h file to the KEYWORD_ARRAY in the pv_params.h file of the picovoice_app app created above.

.

.

Create a new Speech-to-Intent context

.

At Picovoice console, I created a new context call "Door access" using the Empty Picovoice template:

.

.

I added one Slot and two Intents:

.

.

.

.

Trained and downloaded the intent model:

.

.

I unzipped the model and copied the CONTEXT_ARRAY from the Door-access_en_cortexm_v2_1_0.h file to the CONTEXT_ARRAY in the pv_params.h file of the picovoice_app app created above.

.

.

Add the OLED Display functionality

.

Following my earlier post, I added the libraries to the Picovoice app in Eclipse IDE for ModusToolbox. I added the following code to the beginning of the main(void) function in the main.c file:

cy_rslt_t result;
    cyhal_i2c_t i2c_obj;

    /* Initialize the device and board peripherals */
    result = cybsp_init();

    CY_ASSERT(result == CY_RSLT_SUCCESS);

    /* Initialize the I2C to use with the OLED display */
    result = cyhal_i2c_init(&i2c_obj, CYBSP_I2C_SDA, CYBSP_I2C_SCL, NULL);

    CY_ASSERT(result == CY_RSLT_SUCCESS);

    /* Initialize the OLED display */
    result = mtb_ssd1306_init_i2c(&i2c_obj);

    CY_ASSERT(result == CY_RSLT_SUCCESS);

.

In the src/pv_psoc6.c file's pv_board_int(void) function, I commented out the followinig code as it was called in the main(void) function:

//    result = cybsp_init();
//    if (result != CY_RSLT_SUCCESS) {
//        return PV_STATUS_INVALID_STATE;
//    }

.

I have made some change to the inference_callback() function to display the name and passcode on the display:

static void inference_callback(pv_inference_t *inference) {
    cy_rgb_led_on(CY_RGB_LED_COLOR_BLUE, CY_RGB_LED_MAX_BRIGHTNESS);

    printf("{\r\n");
    printf("\tis_understood : '%s',\r\n", (inference->is_understood ? "true" : "false"));
    if (inference->is_understood) {
        printf("\tintent : '%s',\r\n", inference->intent);
        if (inference->num_slots > 0) {
            printf("\tslots : {\r\n");
            for (int32_t i = 0; i < inference->num_slots; i++) {
                printf("\t\t'%s' : '%s',\r\n", inference->slots[i], inference->values[i]);

                if (strcmp(inference->slots[i], "name") == 0) {
                   	strcpy(name, inference->values[i]);
                } else if (strcmp(inference->slots[i], "digit1") == 0) {
                	passcode[0] = toDigit(inference->values[i])+'0';
                } else if (strcmp(inference->slots[i], "digit2") == 0) {
                	passcode[1] = toDigit(inference->values[i])+'0';
            	} else if (strcmp(inference->slots[i], "digit3") == 0) {
            		passcode[2] = toDigit(inference->values[i])+'0';
				} else if (strcmp(inference->slots[i], "digit4") == 0) {
					passcode[3] = toDigit(inference->values[i])+'0';
				}

            }
            printf("\t}\r\n");

            if (strcmp(inference->intent, "passcodeIntent") == 0) {
            	passcode[inference->num_slots] = '\0';
            }

        }
    }
    printf("}\r\n\n");

    displayAt(name, 20);
    displayAt(passcode, 30);

    if (strlen(name) > 0 && strlen(passcode) > 0) {
        strcpy(message, "Access denied");

        for (int i=0; i<NO_USERS; i++) {
			if (strcmp(names[i], name) == 0 && strcmp(passcodes[i], passcode) == 0) {
				strcpy(message, "Access allowed");
				name[0] = '\0';
				passcode[0] = '\0';

				break;
			}
		}

		displayAt(message, 50);
    }

    for (int32_t i = 0; i < 10; i++) {
        if (cy_rgb_led_get_brightness() == 0) {
            cy_rgb_led_set_brightness(CY_RGB_LED_MAX_BRIGHTNESS);
        } else {
            cy_rgb_led_set_brightness(0);
        }
        Cy_SysLib_Delay(30);
    }
    cy_rgb_led_off();

    pv_inference_delete(inference);

}

.

Here are the other helper code with additional variables and helper functions:

static char* names[NO_USERS] = { "peter", "david", "john"};
static char* passcodes[NO_USERS] = { "1234", "2234", "3234" };

static char numbers[10][10] = {"zero", "one" , "two", "three", "four", "five", "six", "seven", "eight", "nine"};

char name[10];
char passcode[10];
char message[20];

int toDigit(const char* word) {
    for (int i = 0; i < 10; i++) {
            /* word to number conversion */
            if (strcmp(word, numbers[i]) == 0) {
                    return i;
            }
    }
    return -1;
}

.

And this is the display function in pv_psoc6.c:

void displayAt(char* msg, int y)
{
    GUI_DispStringAtCEOL(msg, 0, y);
}

.

How It Works

.

The Picovoice app will capture the slots' values of the nameIntent and passcodeIntent. These values will be displayed on the OLED. If the name and passcode matches the values in the names and passcodes array, The app will display the "Access allowed" message, otherwise it will display the "Access denied" message.

Since the inference value is of following format, I wrote toDigit() (seed code above) helper function to carry out the conversion:

{
is_understood : 'true',
intent : 'passcodeIntent',
slots : {
'digit1' : 'one',
'digit2' : 'two',
'digit3' : 'four',
'digit4' : 'five',
}
}

.

The Demo

.

Here is a picture of the Door Access app.

.

And this is a video.

.

Summary

.

This is a very simple Picovoice door access proof-of-concept app that I have made with ModusToolbox. Below are list of enhancements that make app more robust:

• the name and passcode database can be downloaded from the cloud instead of hardcoded in the code;
• another way is doing the authentication by calling a REST API;
• besides displaying the "allowed" message, a lock actuator can be added to the system;
• add additional sophisticated and conversational dialogues to the app.

.