DMI Dog Machine Interface

What to expect?

First of all, I was wondering if it is really true that you can tell something about your dog's health by its nose to make sure it is not an old wife's tail. After Googling and reading for an hour or two, I concluded it is certainly not the best indicator of a dog's health. A dog's nose will be quite different after a nap or after drinking water. A nose can also be very moist if your dog has a 'cold'. I'm no veterinarian, but it should be a fun challenge to measure and monitor it, and that is something I can do (I hope).

Getting started

I had an idea, and I was one of the lucky ones that received the hardware. I read the contest guidelines, and I was confident I would use the ModusToolbox3.0 for the extra points I could receive in the contest. I updated the firmware of my 4100S Pioneer kit and got started. After a few hours, I managed to run the tutorials, but changing the code was a too significant hurdle to take. After all, I have no experience with the PSoC environment.

After a lot of hours, I was completely stuck, and something had to change. I did some more research, and I arrived at the PSoC101 video series and PSoQ creator. I would highly advise doing the tutorials before you start programming on your own.

Breaking down the project

It works best to break down the project into smaller pieces, so I did just that. First, I have to access the sensor data. Then I have to store the sensor data and visualise it to make sure I can detect differences in nose moisture. Then I should set some threshold that triggers an event.

• Accessing the Sensor data
• Reading and visualising the Sensor data
• Testing with the doggo
• Triggering of an event

Getting hold of the sensor data

The Pioneerkit has a proximity sensor loop around the whole front plate, two cap sense buttons and a touchpad. I would like to use the touchpad, but first, I would get started by using one of the Buttons. When reading the documentation and some trial and error, I learned that PSoC is geared up to self-learn or learn the threshold value of the sensor through a user interface application to program the button. What I wanted to do was read the raw sensor data and not make use of any of the fancy firmware.

After reading some of the documentation and reading and some more Googling, I learned you should check the 'Enable Self-test library'.

This will enable you to use the function CapSense_GetSensorCapacitance, which can read the RAW data from the sensor inputs. I first wrote a code to write the capacitance value to the COM port on my computer.

#include "project.h"
#include <stdio.h>
int main(void)
{
    CyGlobalIntEnable; /* Enable global interrupts. */

    /* Place your initialization/startup code here (e.g. MyInst_Start()) */
    
    UART_Start();
    UART_UartPutString("Ready for Doggy");
    CapSense_Start();
    
    CapSense_InitializeAllBaselines();
    
    char str[20];
    
    for(;;)
    {
        /* Place your application code here. */
        CapSense_RunSelfTest(10);
        CapSense_TST_MEASUREMENT_STATUS_ENUM measurementStatus;
        uint32 capacitance = CapSense_GetSensorCapacitance(CapSense_BUTTON0_WDGT_ID, CapSense_BUTTON0_SNS0_ID, &measurementStatus);
           
        sprintf(str, "%lu\n", capacitance);
        UART_UartPutString(str);             
        CyDelay(10); // Delay to just send the value every 10ms. 
    }
}

If I open an empty Arduino project and the COM-port the Pioneer kit is connected to, I get this as the result:

If I press the button, it goes up to 4400+. So I managed to read the raw data and send it to the COM-port. A little experimenting makes me understand the need for the firmware to make it possible to use it as a button. For example, if I restart the controller a day later, I get other values when the button is not pressed. I also noted when you pressed somewhere else on the board, not near the button, the value also changed. I added this basic project as a separate project called RAW_DATA_TO_COM if you want to use it for your own project.

I went thru a lot of trouble to see if I could do the same for the touchpad on the board as a whole, but after a long time reading three of the documentation, I did not find a way to do this. I needed a solution because the button is a too small area for my dog's nose. I then came up with this.

I had a flashcard of an old PLC (Industrial microcontroller) lying around, and I stole a hairband from my girlfriend. The metal case works to enlarge the button, and it works like a charm. Like they say: "If it is stupid, but it works, it isn't stupid".

Reading and visualising the Sensor data

To visualize the data and experiment with it, I wrote the following Python script.

import serial
import time
import matplotlib.pyplot as plt

# Open the serial port
ser = serial.Serial('COM4', baudrate=115200)
print('SerialPort is Read')

# Initialize an empty list to store the sensor data
CAP_Sensor_Data = []

# Read the data from the serial port for 20 seconds
start_time = time.time()
while time.time() - start_time < 20:
    # Read a line of data from the serial port
    line = ser.readline().decode().strip()
    
    # Check if the line is non-empty
    if line:
        # Convert the line to an integer and append it to the list
        CAP_Sensor_Data.append(int(line))

# Subtract the minimum value from each element in the list
min_value = min(CAP_Sensor_Data)
CAP_Sensor_Data = [val - min_value for val in CAP_Sensor_Data]

# Print the list
print(CAP_Sensor_Data)

# Plot the data as a graph
plt.plot(CAP_Sensor_Data)
plt.xlabel('Time')
plt.ylabel('Sensor Data')
plt.title('Sensor Data Over Time')
plt.show()

When the script is started, it will monitor the serial port for 20 seconds and store it in a list. Because the basic number is pretty large (4200<), I first find the smallest number and subtract it from all the values. Then I print the list and visualise it in a graph. This is the output when pressed with my finger:

..... 05, 105, 105, 105, 105, 105, 157, 105, 105, 157, 105, 105, 105, 105, 105, 105, 157, 105, 157, 105, 105, 105, 105, 105, 105, 105, 105, 157, 105, 157, 105, 157, 209, 105, 105, 157, 105, 105, 105, 105, 105, 105, 105, 105, 157, 105, 209, 105, 105, 105, 105, 105, 105, 157, 105, 157, 105, 157, 105, 157, 157, 157, 261, 1147, 1147, 1042, 1042, 938, 1094, 1094, 1199, 1251, 1355, 1251, 1251, 1251, 1042, 1199, 1199, 1147, 1147, 1199, 1251, 1407, 1251, 1303, 1251, 1042, 1094, 990, 782, 157, 209, 157, 157, 105, 105, 105, 105,......

(Just a part of the list)

I saved the Python script as Visualise_CAPSENSE_Sensor.py if you want to try it out yourself.

Testing with the doggo

I mounted the sensor to the place where normally here pushbutton is. I used Double sided tape to connect the card to the sensor and mounted the pioneerkit to the wall.

Time to wake up Sloebie to try it.

The touches she makes are very light, as you can also notice in the graph.

This is where I got really stuck. The touches she makes are really light, and I can not detect any difference if she just had a drink or her nose after playing with a towel. I also did some experiments on how to detect a wet finger or dry finger and found a possible way to detect the difference. If after the touch, some moister is still on the button, there is a slight difference in capacitance.I tried multiple times to achieve this with the Dogo, but I failed at this (for now).

How to help my dog 'speak' to me

“If, at first, you don't succeed, lower your expectations”

So I failed in my initial goal for this project, but I learned a lot on the way and still wanted to do something fun with the PSoC pioneer kit. So my dog uses the bell to indicate she wants to go outside or she wants attention. The DING DONG sounds get really boring really quick, so I wrote some code to enable her to "talk" to us.

I managed to do this by using the example of this PSoC tutorial that I changed to send a signal to the PC running the Python script. I changed the Capsense button tuning to manual and changed the finger threshold to make it very light, so Sloebie could easily annoy us. The project is archived and can be found in the code section under the name 'DOG_HMI'.

This is the main code:

#include "project.h"
#include <stdio.h>


int main(void)
{
    CyGlobalIntEnable; /* Enable global interrupts. */

    /* Place your initialization/startup code here (e.g. MyInst_Start()) */

    UART_Start();
    UART_UartPutString("Ready for Doggy\r\n");

    CapSense_Start();
    CapSense_InitializeAllBaselines(); 
    uint32_t prev_button_state = 0;

    for (;;)
    {
        /* Do this only when a scan is done */
        if (CapSense_NOT_BUSY == CapSense_IsBusy())
        {
            /* Process all widgets */
            CapSense_ProcessAllWidgets();

            /* Scan result verification */
            /* Add any required functionality based on scanning result */

            uint32_t button_state = CapSense_IsWidgetActive(CapSense_BUTTON0_WDGT_ID);

            if (button_state != prev_button_state)
            {
                if (button_state)
                {
                    UART_UartPutString("h\r\n");
                }
                prev_button_state = button_state;
                CyDelay(2000);
            }

            /* Start next scan */
            CapSense_ScanAllWidgets();
        }
    }
}

/* [] END OF FILE */

The Python script reads the COM port, and if a signal ('h') is set triggered by the Pioneerkit the script will create first close the already open music player, create an mp3 file from the text and play it.

import serial
import time
from gtts import gTTS
import os

# Define the text messages that you want
text_messages = [
    'Let me out Human!',
    'Fulfill your oath and let me out',
    'You may think youre the boss, but we both know who really is in charge. Let me out!',
    'Please let me out to catch the squirrels',
    'Come on, my leash is calling my name. Let me out!',
    'Lets go Human, Time to walk!',
    'I have an urgent matter to attend to. Please let me out.',
    'Please, get out the couch and let me out you lazy human',
    'Let me out, you need the exercise',
    'For the love of god, let me out',
    'Let me out, Let me out,Let me out, Let me out,Let me out, Let me out',
    'I swear, if you dont let me out now, I will chew up your shoes!',
    'You know what they say about idle paws, right? Let me out!',
    'I cant hold it any longer, let me out or you will regret it!',
    'If you dont let me out, Ill start barking and the neighbors will know why.',
    'I have an urgent matter to attend to. Please let me out.',
    'I need to go outside to get some inspiration for my next novel. Let me out!'
]


# Set the language and COM port
language = 'en'
COM_Port = 'COM6'

# Initialize a counter variable to keep track of which message to play
counter = 0

# Open the serial port
ser = serial.Serial(COM_Port, baudrate=115200)
print('SerialPort is Read')

# While loop to continuously read the serial port
while True:
    # Read a line of data from the serial port
    line = ser.readline().decode().strip()

    # Check if the received text is 'h'
    if line == 'h':
        os.system("TASKKILL /F /IM wmplayer.exe")
        time.sleep(2)

        # Use the counter variable to select the appropriate text message
        text = text_messages[counter]

        # Generate the speech file using the selected text message
        myobj = gTTS(text=text, lang=language, slow=False)
        myobj.save("Speech.mp3")
        os.system("Speech.mp3")

        # Increment the counter variable to select the next message next time
        counter += 1
        if counter >= len(text_messages):
            counter = 0


# Close the serial port
ser.close()

Please note you need to be connected to Wifi to make the connection to the text-to-speak API.

Conclusion

The thing I set out to do was not possible for me, but I learned a lot on the way and had fun doing the second project that my Dogo Sloebie and I would count as a greater success for dogs and humankind.