Smart grain portioner for visually impaired people

Overview

For the visually impaired, cooking is often a difficult task, many recipes require specific amounts of ingredients to make a dish, many cooking utensils and appliances are not designed with accessibility for the blind in mind. Also, portioning ingredients can be very difficult because you cannot see the measurement markings on the cookware. On the other hand, identifying ingredients in the kitchen becomes another challenge as most cereals when purchased come in bags of the same weight and without any braille markings to differentiate them.

• The lack of ability to visually identify ingredients in the kitchen can make it difficult to execute a recipe efficiently.
• Spoilage of food and recipes due to tampering by detecting the ingredient by touch.
• Lack of utensils for the visually impaired may deter some people from participating in culinary activities.
• Lack of inclusive utensils to assist in portioning food results in recipes that do not taste good due to too many or too few ingredients.

Build2gether 2.0 Challenge

This project was developed to participate in the Build2Gether 2.0 competition, which encourages the creation of innovative technological solutions aimed at improving the quality of life of people with disabilities.

This year's challenge is organized into two main areas, each divided into two specific tracks to guide the innovations.

VISUAL IMPAIRMENTS 👨‍🦯👩‍🦯

• Track 1: Adaptation for OUTDOOR Activities for People with Visual Impairments
• Track 2: Adaptation for INDOOR Activities for People with Visual Impairments

MOBILITY IMPAIRMENTS 👨‍🦼👩‍🦼

• Track 1: Accessible HOME & TOOLS for People with Mobility Impairments
• Track 2: Accessible SPORTS & HOBBIES for People with Mobility Impairments

To address the challenge of Track 2 for the visually impaired, which focuses on adapting indoor activities, the following technological solution was presented:

To solve this problem an intelligent grain portioner was manufactured, this portioner has 6 compartments for different types of grain and is voice controlled. When executing the command "white rice" the device will rotate the compartments to select the corresponding compartment, then the system opens the gate where it drops the amount corresponding to a cup of rice (Aprox 200 gr).

To build this device we used an Arduino Nano RP2040 Connect, a stepper motor with its respective controller and a servomotor in charge of opening and closing the gate. In the software section, the Arduino IDE and the Edge Impulse platform were used to create the voice command recognition algorithm. This device uses TinyMl techniques so that it can be used without internet connection, so it can also be used in vulnerable areas that do not have internet coverage.

How to Working

Hardware:

Arduino Nano RP2040 Connect:

The Arduino Nano RP2040 Connect has the same microcontroller as the Raspberry Pi RP2040. It also takes advantage of the 32-bit Arm® Cortex®-M0+ dual-core to realize Internet of Things projects with Bluetooth® and WiFi connectivity thanks to the U-blox® Nina W102 module. In addition, it has a LSM6DSOXTR accelerometer-gyroscope sensor, RGB LED and an integrated MP34DT06JTR microphone.

The MP34DT06JTR sensor is a ultra-compact microphone that use PDM (Pulse-Density Modulation) to represent an analog signal with a binary signal. The sensor's range of different values are the following:

• Signal-to-noise ratio: 64dB
• Sensitivity: -26dBFS ±3dB
• Temperature range: -40 to 85°C

How can you build it :

To build this system, we must follow these steps:

Step One: Breadboard diagram

Step Two: Clone Edge Impulse project

In this project, 6 of the most commonly used grains and cereals in the basic food basket in Latin America were selected. These words were carefully chosen to ensure high precision and minimize the probability of false positives. the keywords were: Rice, Beans, lentils, Oats, Cornand Cereal.

Access Edge Impulse, log in with your account, access the project "Smart_grain_portioner_Buil2gether_2.0" and click "Clone project" to duplicate it in your account. Then, open the cloned project, go to the "Deployment" tab, select "Arduino library" and click "Build" to download the generated files.

If you want to create a new model with different grains and cereals, I invite you to follow the tutorial "Responding to your voice" from Edge Impulse. This tutorial was used to develop the machine learning model used in this system, providing detailed guidance for capturing data, training the model and deploying it effectively. By following this tutorial, you will be able to tailor the system to your specific needs, ensuring that it works correctly with the new grain and cereal types.

Step Two: Import the model into Arduino ID

If you don't have the Arduino IDE installed, download it from Arduino and install it on your computer. Run the Arduino IDE, go to Sketch > Include Library > Add.ZIP Library, navigate to the location of the downloaded Edge Impulse files and select the ZIP file to add it as a new library.

Step Three: Project Files

The system requires the model previously downloaded from Edge Impulse and the main.ino main program.

1. Download and unzip the project bundle from the following link on GitHub: Smart-grain-portioner-for-visually-impaired-people-main.zip.

2. In the Arduino IDE, go to Files > Open. navigate to the download folder, find and select the main project file "Main.ino".

3. In the Arduino IDE, go to Sketch > Include library > and select the "Project_name_Inferencing"

4. Download and install the Mbed OS Nano Board package through the Arduino IDE Board Manager. Then, in the Tools > Board menu, select the Arduino Nano RP2040 Connect board. Connect your Arduino to the computer using a USB cable and choose the correct port under Tools > Port. Finally, click the Upload button to compile and upload the program to the Arduino Nano RP2040 Connect board.

Step Five: System testing

Step Six: Model testing