Wi-Fi Voice Controlled Robot Using Google Assistant

In this tutorial, I show you how to make a voice-controlled robotic tank controlled using a smartphone, Google Assistant, IFTTT, and Adafruit.io platforms. An ESP8266 Wemos D1 board was used in this project. It receives commands from a Wi-Fi network and control the speed of the motors. Other development boards might also be used (NodeMCU, Firebeetle, etc.), and the principles presented in this tutorial can be applied on other models of robots and other IoT gadgets.

I've previously designed a Blynk controlled robotic tank. It connects to a Wi-Fi network and receives commands from Blynk server. A smartphone running Blynk app was used as a remote contro, and different input methods were used: push buttons, sliding bars and even smartphone's accelerometer. You can find more about this project here: https://www.hackster.Io/IgorF2/wI-fI-controlled-robot-usIng-wemos-d1-esp8266-and-blynk-464198

This time I decided to try a different way to control the same robot: voice commands. It might be usefull if you want to remotelly control a robot without using your hands, or if you want to made it accessible for someone with limited movements. One might think of a robotic voice controlled wheel chair, for instance.

A DIY robotic kit was used, along with some of my favourite tools: Adafruit.io, IFTTT and Arduino IDE. You can also use different kit or even desing your own robot using simple materials, without the need of using complex tools such as 3D printers and laser cutting machines. You can find an example on one of my previous tutorial: https://www.hackster.io/igorF2/widc-wi-fi-controlled-fpv-robot-8f1e09

There are several ways to use this tutorial. You can use it to:

• Learn how to program an ESP8266 using the Arduino IDE;

• Practice your electronics and soldering skills;

• Learn how to assemble a robotic kit;

• Learn how to use Adafruit.io along with IFTTT in your projects;

Liked this project? Please consider supporting my future projects with a small Bitcoin donation! :D

BTC Deposit Address: 1FiWFYSjRaL7sLdr5wr6h86QkMA6pQxkXJ

Step 1: Tools and Materials

The following tools were used in this project:

• Solder iron and wire. The DC motors already came with wires soldered to its terminals... But it will eventually break and you might have to resolder it. So consider having a good solder iron and wire neaby.

• EVA foam sheet (or other non-conductive material). The robot chassis I used in this project is made of aluminum, and circuit boards are installed on this metal parts. I used a layer of foam sheet between the boards and the metal plate to avoid possible short-circuits.

• Double sided tape. It was used for glueing the foam sheets to the circuit boards, and for the installation of the H-Bridge modue.

• Scissors, for cutting some foam sheet rectangles.

I used the following hardware parts for my project:

• Wemos D1 ESP8266 dev board (link). Wemos D1 board is really easy to use and program with Arduino IDE. It has the same footprint of and ordinary Arduino Uno! This way most of Arduino shield will also work with this board. It has built-in Wi-Fi module, so you can use it in a variaty of projects.You can also use other ESP8266 based boards (link / link).

• L298N dual channel H-bridge module (link). This module allows the 3.3V signals from the Wemos (or an Arduino) to be amplified to the 12V needed for the motors.

• DIY Robot Chassis Tank (link). This awesome kit has with everything you need to built a tank: two DC motors, gears, tracks, bolts, nuts, etc. It already comes with the tools need for assembling the chassis, which is great for beginners!

• 18650 3.7V batteries (x3) (link). I used to power the whole circuit. This tank uses 12V motors. I used three 3.7V batteries in series for powering them.

• 3S 18650 battery holder (link). It can hold three 18650 batteries in serie, and can be easilly be attached to the back of the tank.

• 18650 battery charger (link). Your batteries will eventually run out of power. When that happens, a battery charger will come to your rescue.

• Jumpers (link). I used 6 male-female jumpers for signals between the h-bridge an the Wemos, and 2 male-male jumpers for 5V and Gnd. You might need more if you plan to add some sensors.

• Micro USB cable. You'll need this for uploading your code. Most of the boards already come with its own cable.

The links above are only a suggestion of where you can find the items used in this tutorial (and maybe support my future tutorials). Feel free to search for them elsewhere and buy at your favorite local or online store.

Step 2: Assembling the Robot

The first part of this project was the assembly of the robot structure.

In previous projects I developed the structure of my own robot, using easily accessible materials (without the need for complex tools, 3D printers or laser cutting machines). You can find this project in the link below:

https://www.hackster.io/igorF2/widc-wi-fi-controlled-fpv-robot-8f1e09

This time I decided to give a chance for a robotic kit obtained from an online store. You can find a link to the robot chassis at the following link: http://bit.ly/2ycI8fP. If you're looking for a kit, I think it's a good option!.

At first it seemed that the assembly would be complex or that I would run into problems like the lack of parts (given the amount of parts that make up the kit). But this kit really surprised me! All the pieces seem to me of good quality, and several spare parts accompanied the kit. Thus, a screw lost underneath the workbench won't make it impossible to carry out your project, which I found to be excellent (especially after losing a couple of screws).

Another positive point is that all the tools needed to mount the robot are included in the kit (a couple of Allen wrenches and the screwdriver). I believe this makes the kit excellent for starters who doesn't have lots of tools!

As a negative aspect I would highlight the lack of documentation. The robot's assembly manual (a spreadsheet file in Chinese) is not extremely user friendly, and there isn't a lot of tutorials online for that tank. And so I decided to document the process of assembling my robot in the video above! Another point of attention concerns the material of the robot structure. The base is entirely made of aluminum, which can cause some short circuit if the pins of the printed circuit boards touch the frame.

There are other online kits that you can use. You can even maker your own structure, as I described above.

Step 3: Wiring Up the Circuit (Wemos D1)

In the video above I show you how you can assemble the electronics in the robot structure.

A power pack made of three 18650 batteries was installed on the back of the robot. It provides 11.1V (3 x 3.7V) to the robot. That's enough to power the 12V DC motors.

A L298N dual channel H-bridge was used for the control of the motors. It receives some 3.3V signals of the Wemos board, and provide higher voltages for the motors. It also allow the motors to run in both directions, depenting on the combination of those input signals.

Each device was connected according to the schematics.

Follows a list of the pins you'll need to connect:

Wemos D1 inputs/outputs:

• Digital pin D3 (GPIO5) => H-Bridge ENB pin

• Digital pin D4 (GPIO4) => H-Bridge IN4 pin

• Digital pin D5 (GPIO14) => H-Bridge IN3 pin

• Digital pin D6 (GPIO12) => H-Bridge IN2 pin

• Digital pin D7 (GPIO13) => H-Bridge IN1 pin

• Digital pin D8 (GPIO0) => H-Bridge ENA pin

• 5V pin => H-Bridge 5V pin

• Gnd pin => H-bridge Gnd pin

H-Bridge inputs/outputs:

• ENB pin => Wemos D3 pin

• IN4 pin => Wemos D4 pin

• IN3 pin => Wemos D5 pin

• IN2 pin => Wemos D6 pin

• IN1 pin => Wemos D7 pin

• ENA pin => Wemos D8 pin

• 5V pin => Wemos 5V pin

• Gnd pin => Wemos Gnd pin

• Gnd pin => Battery pack negative wire

• 12V pin => Battery pack positive wire

• OUT1 => Right motor negative wire

• OUT2 => Right motor positive wire

• OUT3 => Left motor positive wire

• OUT4 => Left motor negative wire

Step 4: Set Up ESP8266 Board on Arduino IDE

For this project I used Arduino IDE for programming my Wemos. It's the easier way if you've already used an Arduino before, and you won't need to learn a new programming language, like Python or Lua for instance.

If you've never done this before, first you'll have to add ESP8266 board support to the Arduino software.

1. Download and install Arduino IDE latest version

You can find the latest version for Windows, Linux or MAC OSX on Arduino's website: https://www.arduino.cc/en/main/software

Download it for free, install it on your computer and launch it.

2. Adding ESP8266 board

Arduino IDE already comes with support to a lot of different boards: Arduino Nano, Mine, Uno, Mega, Yún, etc. Unfortunatly ESP8266 isn't by default among those suported development boards. So in order to upload your codes to a ESP8266 base board, you'll have to add its properties to Arduino's software first.

Navigate to File > Preferences (Ctrl +, on Windows OS);

• Add the following URL to Additional Boards Manager textbox (the one on the bottom of the Preferences window):

http://arduino.esp8266.com/stable/package_esp8266com_index.json

Navigate for Tools > Board > Boards Manager for adding your ESP8266 board.

• If the text box wasn't blank, it means had already add other boards before on Arduino IDE before. Add a comma at the end of the previous URL and the one above.

• Hit "Ok" button and close the Preferences Window.

• Type "ESP8266" on the search text box, select "esp8266 by ESP8266 Community" and install it.

Now your Arduino IDE will be ready to work with a lot of ESP8266 based development boards, like the generic ESP8266, NodeMcu (which I used in this tutorial), Adafruit Huzzah, Sparkfun Thing, WeMos, etc.

3. Adding the libraries

The following libraries will be used for our Arduino code:

• Arduino http client library (https://github.com/arduino-libraries/ArduinoHttpClient)

• Arduino IO library (https://github.com/adafruit/Adafruit_IO_Arduino)

• Adafruit MQTT library (https://github.com/adafruit/Adafruit_MQTT_Library)

Navigate to Sketch-> Include Library -> Manage Libraries... on your Arduino IDE and add the libraries above.

............

Now that your dev environment is ready, let's move on to the next step!

Step 5: Adafruit.IO Configuration

There are a lot of datalogging services available for communicating a microcontroller to the web. With those services you can upload/download data to/from the cloud, and do a lot of cool stuff. Take a look on my tutorial on how to use an Arduino + ESP8266 to send data from a mini-weather station for Thinkgspeak for instance (link).

Adafruit.IO is one of those free services. It's really easy to use and promises to bring internet of things to everyone!

Create Adafruit IO Web Feed

Under Feeds >Create a new Feed add a new feed named "voice commands". It will create a database, and we will use it store the commands received by the gadget.

• Sign in at https://io.adafruit.com/

On the next step I will show you how to configure IFTTT, another platform I used in this project. The idea here is simple: IFTTT will have some triggers configured and send some data to Adafruit.IO platform when a given logic is true. The gadget will be able to read the data stored in a given feed on Adafruit.IO, execute some logic and perform some actions.

It's also a good time to copy your Adafruit.IO key, which will be later used for allowing your device accessing the database. Navigate for Settings > View AIO key and copy the active key code. You'll need it for your Arduino (NodeMCU) code on next steps.

Step 6: IFTTT Configuration

IFTTT is a free platform that helps you connects apps and devices. You can use it to connect your smartphone with other gadgets, or to share data between your favourite webservices (like Google, Facebook, Twitter, Instragram, etc.) and other physical devices, for instance. And the best part is that it's really easy to use!

IFTTT uses a "if this then that" logic, where "this" represents a service that will trigger a given action given by "that". This way you create small applets connecting webservices and devices. For the project described in this tutorial, I used Google Assistant to send voice commands from a smartphone to Adafruit.io, which is then received by the robot.

First you'll have to sign in at:

https://ifttt.com/

Then install IFTTT app on your smartphone. You can find it at Google Play Store:

https://play.google.com/store/apps/details?id=com.ifttt.ifttt

https://itunes.apple.com/us/app/ifttt/id660944635?mt=8

On the website, navigate to New Applet (click the arrow button next to your login to access the menu).

In the next step I'll show you how to create IFTTT applets.

Step 7: Install Google Assistant

For this project, you'll also need to install Google Assistant App on your Android smartphone or iPhone. You can find it on Google Play or Apple Store:

https://play.google.com/store/apps/details?id=com.google.android.apps.googleassistant&hl=en

https://itunes.apple.com/br/app/google-assistant/id1220976145?l=en&mt=8

Google Assistant is a virtual assistant developed by Google. It's artificial intelligence is able identify voice commands, perform seaches and other tasks. It has a great integration with IFTTT service, which was explored in this tutorial.

Step 8: IFTTT Applet

For this applet we will create a voice command that will make the robot move in a given direction or turn left/right.

Create the applet on the website:

1 - Click +This;

2 - Type "Google Assistant" on Seach service text box and select Google Assistant widget;

3 - Choose "Say a phrase with both a number and a text ingredient" trigger. As it's described on IFTTT website, it will create a trigger that fires every time you say "Ok Google" followed by a phrase with a text ingredient and a number ingredient;

4 - To complete the trigger, you'll have to define the phrase you want to say. IFTTT allows you to configure more than one phrase as a trigger. $ represent the text ingredient and # the number ingredient. In my case, I define the following commands:

• Turn $ # degrees

• Rotate $ # degrees

• Move $ #

5 - You'll also have to define the message it will say in response if it understands you command. In my case I configured it to Say "Turning $".

6 - Now choose +That;

7 - Type "adafruit" and select Adafruit > Send data to Adafruit IO. This will send data to a feed in your Adafruit IO account whenever the trigger you configured previously (+This) is activated;

8 - Configure Feed name as "voice command". In "Data to save" choose "Add ingredient" and add the TextField and NumberField. Add ":" between then. This will work as a delimiter later on Arduino code.

9 - Finish your applet and turn it on.

Now whenever you say one of the phrase (for instance: "Turn left 90 degrees" or "Move forward 10"), it will store the command and the number on Adafruit.io feed (in previous example, "left:90" or "forward:10" will be stored).

Repeate previous instructions if you want to create different phrases.

In the following step I'll show you how to make the code the ESP8266 to perform an action when the command is received.

Step 9: Arduino Code

Download the code and open it on Arduino IDE.

You'll have to update the following parameters before uploading the code:

• IO_USERNAME = Adafruit.io user name

• IO_KEY = Adafruit.io key

• WIFI_SSID = Wi-Fi network SSID

• WIFI_PASS = Wi-Fi network password

Select 'Wemos D1 R2 & mini' (if you're using this board), plug it on your computer's USB port and upload the code. It will take a while (much more than compliling and uploading a sketch for an Arduino... be patient...). Now it's a good time for you to give a like on this tutorial while you wait! :D

When upload is completed, unplug the USB cable, and put the batteries. The code will start running immediatly and the Wemos board will automatically try to connect the Wi-Fi router and wait for the commands from Adafrui.io server.

Step 10: Usage

Robot ready, now it's time to try it! Launch Google Assistant on your phone. You may press the microphone icon on the app or just Say "Ok Google" and it will start listening your commands.

Say on of the phrases you configured before (Move [forward/back] [number] or Turn [right/left] [number]). If Google Assistant understands what you've just said, it will send a message to an Adafruit.io feed (using IFTTT app). ESP8266 board will listen to that feed and if a new message arrives, it will split it in two parts (a command and a number).

Command string might assume one of the following values: left, front, back or right, and will be used to select appropriate direction and speed of each track. Number part might represent the rotation angle (if 'left' or 'right' command was received) or the distance it has to move (if 'forward' or 'back' command was received).

Notice my robot uses an open-loop controller. This way, angles and distances won't be precise (and might even deppend on how much power there's left on the batteries). If you need precision on the movements, you'll have to add a closed-loop system (measure the rotation of each motor, for instance, and use it as a feedback).

Step 11: Final Considerations

I hope this project was as clear and educational as possible. If you liked it, please don't forget to 'like' and 'share' it with your friends. Your support is very important! :D

If you still don't follow my tutorials, please take a look at those other projects. I hope you like them!

https://www.hackster.Io/IgorF2/joy-robot-robo-da-alegrIa-bba54f

https://www.hackster.Io/IgorF2/wI-fI-controlled-robot-usIng-wemos-d1-esp8266-and-blynk-464198

https://www.hackster.Io/IgorF2/wI-fI-controlled-robot-usIng-wemos-d1-esp8266-and-blynk-464198

https://www.hackster.Io/IgorF2/wI-fI-browser-controlled-robotIc-arm-89b63d

Please consider supporting my future projects with a small Bitcoin donation! :D

BTC Deposit Address: 1FiWFYSjRaL7sLdr5wr6h86QkMA6pQxkXJ