Make your Arduino Nicla Vision SPEAK for your IoT and TinyML

# About the project

This project aims to give the capability of Arduino Nicla vision, or boards like this, to speak. Here, we will use Wi-Fi as the medium to send the information to a computer, using a Python library. Part of the challenge you may face is to update the wifi firmware in Arduino Nicla Vision, it is a typical step if you use Edge Impulse, here you have to upload the Edge Impulse firmware(it is another firmware) for Arduino Nicla Vision and this firmware erases the wi-fi firmware in Arduino, so when you want to deploy your final version is necessary to recover the Wi-Fi firmware to succeed in.

# Introduction

Arduino Nicla vision is a very powerful device, it contains many sensors to make it easy to acquire data and play making deep learning models for the inferring in this device. Mainly, Arduino Nicla Vision is used in TinyML and IoT projects, where in those areas part of edge computing is to manage the consumption of battery to be more autonomous and do not require constant charge it. Because of that, Arduino Nicla Vision or others like this don’t have something that may consume too much energy like speakers, many LEDs, or others.

On the other hand, Arduino Nicla Vision communicates wireless by Bluetooth and Wifi, talking about Wi-Fi has many advantages such as to connect in a private network to send images that it may capture, but Wi-Fi is tricky because it needs to work with firmware pre-installed there, but when you use other software such an Edge Impulse, it erases the memory flash and needs to upload another firmware, this erase also Wi-Fi firmware so when you upload an Arduino code using Wi-Fi library you may encounter with errors. To prevent errors after uploading the Edge Impulse firmware, you have to re-upload the Arduino Wi-Fi firmware.

# Process

Data Acquisition and Upload to Edge Impulse:

• Clone the project from the Edge Impulse public repository. https://studio.edgeimpulse.com/public/494730/live Link
• Ensure the Wi-Fi firmware is fixed in Arduino for seamless data upload.
• Data Acquisition and Upload to Edge Impulse:Clone the project from the Edge Impulse public repository. https://studio.edgeimpulse.com/public/494730/live LinkEnsure the Wi-Fi firmware is fixed in Arduino for seamless data upload.

Development of Python Code:

• Implement MQTT using the paho-mqtt library to handle message queuing and delivery.
• Utilize GTTS (gtts) for text-to-speech conversion to enable audible feedback.
• Development of Python Code:Implement MQTT using the paho-mqtt library to handle message queuing and delivery.Utilize GTTS (gtts) for text-to-speech conversion to enable audible feedback.

Adquiring data from MQTT

By following these steps and utilizing the provided resources, developers can create innovative communication aids that significantly improve the quality of life for deafblind individuals. This solution not only addresses the immediate communication needs but also paves the way for further advancements in assistive technologies, promoting greater inclusion and independence within the deafblind community.

# Would you like to try?

upload this code to Arduino arduino_code.ino (It needs arduino_secrets.h, it is in the folder), then run python_code.py (firstly install all libraries that may needed) You can use this line code in your terminal:- $ mosquitto_sub -t "tinyml-brasil/data" -h "test.mosquitto.org"