Arduino Kitchen Assistant is a device that helps you around the kitchen, by showing recipes, setting timers, displaying temperature of the food, leaving notes, etc. It is powered with the Amazon Alexa and is portable so you can carry it around the kitchen and does not require to be plugged in.
ActivitiesBefore we dive right into building the project, here is how you would interact with this.
- Setting Timers - "Alexa, set a timer for 30 seconds"
- Recipes - "Alexa, how do we cook tea"
- Metric Conversion - "Alexa, 10 ounces to liters"
- Notes - "Alexa, Leave a note to get some milk later"
- Delete Notes - "Alexa, delete all notes"
- Cooking Temperature - "Alexa what is the temperature of the dish"
These are the few utterances you need to get started with the project, we will see more utterances later.
Note:- The skill I built for this project is currently undergoing publication and should available on the app store soon.
Let's Start BuildingThere are four major stages of the project we need to build, the Alexa Skills Kit, Lambda Function, Nodejs Application and the Kitchen Assistant.
The Alexa Skills Kit is the tool/software where we train our Alexa skill to respond to the user input. Where each sentence said by the user is termed as an Utterance and each utterance had an interlinked intent.
After the skills kit comes the AWS Lambda function, this is Nodejs based app which runs on AWS Lambda. This application is used to perform api requests to the Nodejs Application based on the intent and the slot values from the skills kit.
This is a local Nodejs application that can be deployed on a raspberry pi or a laptop. This is where all the data is stored and processed. It also provides what data needs to be displayed on the kitchen assistant or the time delay for the timer.
And the last stage is the kitchen assistant, which gets its data from the Nodejs application of the previous stage.
HardwareFirst, lets start with building the hardware, at the heart of this project is an Arduino Nano and a Esp8266-12 to connect to the internet. All the text is displayed on a 16x2 LCD display. The arduino talks to the esp8266 via serial and the esp8266-12 helps the Arduino Nano connect to the internet (to the nodejs server). The circuit can be found below, if you are not familiar with soldering, you can build this project on a breadboard.
I soldered the circuit on a perfboard and then cut the board to fit the PVC enclosing. For the enclosing I used a PVC box with dimensions of 4 inches x 4 inches.
To hold the LCD display I printed a 3D printed case which fits the 16x2 LCD perfectly and you can use screws to hold the LCD in place. You can go with an I2C LCD driver if you want to skip on soldering the extra wires.
I wanted the project to be portable and I decided to be powered via Li-ion battery, the Li-ion battery is only capable of providing 3.7V hence you will need a boost converter to raise the voltage to 6V to power the ESP8266 and the Arduino Nano.
For the buck converter I used the XL6009 IC which is 10W buck converter and for charging the batteries I use a TP4056 module, which can provide charging currents of up to 1A and has onboard over charging protection.
After all the connections are done, you can power on the circuit and upload the test code (Hello World Program) found below, at first you should see nothing on the screen you may need to vary the 10K pot to adjust the contrast of the display.
// include the library code:
#include <LiquidCrystal.h>
// initialize the library with the numbers of the interface pins
LiquidCrystal lcd(7, 6, 5, 4, 3, 2);
void setup() {
// set up the LCD's number of columns and rows:
lcd.begin(16, 2);
// Print a message to the LCD.
lcd.print("hello, world!");
}
void loop() {
// set the cursor to column 0, line 1
// (note: line 1 is the second row, since counting begins with 0):
lcd.setCursor(0, 1);
// print the number of seconds since reset:
lcd.print(millis() / 1000);
}
This project uses a nodejs server to store all the data, you can either run this on a raspberry pi (You can skip this step if you are using a raspberry pi) or on a PC.
You can download and install the Nodejs software from the official website. Nodejs comes with its own package manager called "npm". To get the server running you will need to install additional dependency called "Expressjs" and for metric conversions you will need a dependency called "convert-units". To install both the dependencies, run the blow commands.
npm install express --save
npm install convert-units --save
After you have installed express, create a file called app.js and copy - paste in the code below you can use any text editor of your preference to do that.
var fs = require('fs');
var app = express();
var newnote =[];
var temp = 0.0;
var time = 0;
var recipies = "0";
var conversion = {"liters":"l","mililiters":"ml","pounds":"lb","ounces":"oz","kilograms":"kg"}
//API to add and delete notes
var convert = require('convert-units')
app.get('/addnote', function (req, res) {
if(req.query.q != null){
var send_data = {};
send_data["status"] = 1;
newnote.unshift(req.query.q);
res.send(send_data);
}
});
app.get('/allnotes', function (req, res) {
var send_data = {};
send_data["note"] = newnote[0];
send_data["status"] = 1;
res.send(send_data);
});
app.get('/deletenotes', function (req, res) {
var send_data = {};
newnote=[];
send_data["status"] = 1;
res.send(send_data);
});
//API for recipies
app.get('/recp', function (req, res) {
var send_data = {};
send_data["recipies"] = recipies;
send_data["status"] = 1;
res.send(send_data);
});
app.get('/newrecp', function (req, res) {
recipies = req.query.q;
if(fs.existsSync(recipies+".json")){
var send_data = {};
send_data["status"] = 1;
res.send(send_data);
}
else{
var send_data = {};
send_data["status"] = 0;
res.send(send_data);
}
});
app.get('/startrecp', function (req, res) {
var contents = fs.readFileSync(recipies+".json");
var jsonContent = JSON.parse(contents);
jsonContent['status'] = 1;
res.send(jsonContent);
});
//API to set timer
app.get('/settime', function (req, res) {
if(req.query.q !=null){
time = req.query.q;
var send_data = {};
send_data["status"] = 1;
res.send(send_data);
}
});
app.get('/readtime', function (req, res) {
var send_data = {};
send_data["status"] = 1;
send_data["time"] = time;
res.send(send_data);
});
//Api to read temperature
app.get('/settemp', function (req, res) {
if(req.query.q !=null){
var send_data = {};
send_data["status"] = 1;
res.send(send_data);
temp = req.query.q;
}
});
app.get('/readtemp', function (req, res) {
var send_data = {};
send_data["status"] = 1;
send_data["temp"] = temp;
res.send(send_data);
});
app.get('/metric', function (req, res) {
var val = req.query.val;
var to = req.query.to;
var from = req.query.from;
var send_data = {};
send_data["status"] = 1;
send_data["data"] = convert(parseInt(val)).from(conversion[from]).to(conversion[to]);
res.send(send_data);
});
app.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
After you create the file you can run the server by running -
node app.js
Make sure you note the IP address of the nodejs server you are running, you will need it later.
Running Nodejs Server on a Raspberry PiTo run the Nodejs server on a raspberry pi you need to download Raspbian Lite and write it to a micro SD Card. After installing Raspbian create a file "ssh" in the root of the SD card to gain ssh access to your Pi.
Once you have the ssh terminal open you should login into the pi the default username is "pi" and the default password is "raspberry".
After logging in lets install nodejs to run our application to do that run the below commands.
sudo apt-get update
sudo apt-get dist-upgrade
curl -sL https://deb.nodesource.com/setup_9.x | sudo -E bash -
sudo apt-get install -y nodejs
Once all the commands have been executed you can check if node js is successfully installed by running.
node -v
This command displays the version of the Nodejs installed. After installing Nodejs you can follow the previous step to get the server running.
MicroPythonNow lets flash the program on the esp8266-12 board, for this project we will be using Micropython firmware on the esp8266. Micropython allows us to run python scripts on the esp8266 and also allows us to use different modules and multi-file programming. Micropython also comes with an inbuilt module to interface with the I/O pins and different communication standards.
To upload micropython on the esp8266-12 you will need to connect it to an FTDI Module, make sure the jumper on the FTDI Chip is set to 3.3V.
Next, you will need to download the latest micropython firmware, to upload the firmware onto the board you will need a python tool called esptool. You can install that by installing python and then running the below command on a terminal window.
pip install esptool
Now that the tool is installed, you can flash the mircopython firmware by typing the following command. Make sure you enter the serial port and the filename with the ones you are using.
esptool.py --port serial_port --baud 460800 write_flash --flash_size=detect 0 file_name.bin
After you have flashed the new firmware you should be able to get a python terminal over the serial port, the baud rate for the serial port is 115200.
Now it is time to upload files on to the flash memory of the esp8266-12, the easiest way to upload files on to the board is via the WEB-REPL, to enable the WEB-REPL run the following command in the serial terminal.
import webrepl_setup
Once that is done, you can upload files onto the board by navigating to the webrepl page and then type in your IP address of the esp8266-12 board.
By default the file which Micropython runs at boot is boot.py file. You can copy the code for the esp8266 below and replace that with the contents in the boot.py file.
# This file is executed on every boot (including wake-boot from deepsleep)
#import esp
#esp.osdebug(None)
import gc
import webrepl
webrepl.start()
gc.collect()
from machine import UART, Pin
import time
import urequests as requests
import ujson as json
server_url = "Your Ip Goes Here" #Replace with your IP of your nodejs server
uart = UART(0, 9600)
uart.init(9600, bits=8, parity=None, stop=1)
pin = Pin(2, Pin.OUT)
def serial_write(text):
uart.write(str(text))
def get_data(url):
r = requests.get(url)
return r.text
def wait_timer(sec):
time.sleep(int(sec))
def get_note():
r = requests.get(server_url+"/allnotes")
data = json.loads(r.text)
return data["note"]
def run_recp():
r = requests.get(server_url+"/startrecp")
data = json.loads(r.text)
for i in range(1,len(data)):
serial_write(data[str(i)]["read"])
wait_timer(3)
r = requests.get(server_url+"/newrecp?q=0")
def check_recp():
r = requests.get(server_url+"/recp")
data = json.loads(r.text)
if data["recipies"] != "0":
run_recp()
def beep():
pin.value(1)
time.sleep(2)
pin.value(0)
def check_timer():
r = requests.get(server_url+"/readtime")
data = json.loads(r.text)
if int(data['time']) > 0:
serial_write("Timer for "+str(data['time'])+"sec")
wait_timer(int(data['time']))
serial_write("Timer Complete")
beep()
wait_timer(3)
r = requests.get(server_url+"/settime?q=0")
return 1
while(1):
serial_write(get_note())
check_timer()
check_recp()
wait_timer(1)
You will need to add your server IP address that you found in the previous step in the line where it says server_url and upload the file. Reset the board and you have successfully programmed the esp8266.
Programming Arduino NanoProgramming the arduino Nano is simple all you need to do is download the arduino IDE and Copy the code from below and paste it in the IDE. Make sure you have selected the right serial port of the arduino nano.
You will need the Adafruit LCD Library to compile this code, if you don't already have it you can install it in the library manager of the arduino IDE.
// include the library code:
#include <LiquidCrystal.h>
// initialize the library with the numbers of the interface pins
LiquidCrystal lcd(7, 6, 5, 4, 3, 2);
void setup() {
// set up the LCD's number of columns and rows:
lcd.begin(16, 2);
// initialize the serial communications:
Serial.begin(9600);
}
void loop() {
// when characters arrive over the serial port...
if (Serial.available()) {
// wait a bit for the entire message to arrive
delay(100);
// clear the screen
lcd.clear();
// read all the available characters
while (Serial.available() > 0) {
// display each character to the LCD
lcd.write(Serial.read());
}
}
}
If you are getting errors while programming the arduino nano remove the connection to esp8266 while programming the nano.
Voice User Interface (VUI)Before we dive right into building the skill we need to plot a VUI diagram for how our skill will interact with the user. The picture below shows all the utterances and responses that Alexa generates for our skill to function with.
Let's start with building the Alexa sill, to start with this you will need an amazon developer account, you can register for a new account or use the same account you use to shop on amazon.
Once you are logged in navigate to Alexa tab and select the amazon skills kit, it is used to develop and publish the skills. The Alexa voice kit is used to develop devices like the echo, example the Sonos device which features Alexa.
Once you are on the skills kit page, select create new skill, you will now be asked to fill in the skill information.
- Skill Type - Select Custom Interaction Model
- Language - Select the Language you want the skill to be based on.
- Name - This is the name of the skill, you can give any custom name.
Invocation Name - This is the name you ask Alexa to open your skill, example "Alexa, open Kitchen Assistant". Where "Kitchen Assistant" is the invocation name.
Next for creating an interaction model we will use the skill builder. You can either create your own intents and slots or the use the one I created by copying the code from below and pasting it in the code editor. After doing that hit Build Model to create your interaction model.
{
"languageModel": {
"types": [
{
"name": "from_metric_type",
"values": [
{
"id": null,
"name": {
"value": "ml",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "liters",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "ounces",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "killogram",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "kg",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "pounds",
"synonyms": []
}
}
]
},
{
"name": "metric_value_type",
"values": [
{
"id": null,
"name": {
"value": "10",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "20",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "3.2",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "4.5",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "5",
"synonyms": []
}
}
]
},
{
"name": "name",
"values": [
{
"id": null,
"name": {
"value": "Poched Egg",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "tea",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "Omlet",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "Pan Cake",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "Cake",
"synonyms": []
}
}
]
},
{
"name": "task_type",
"values": [
{
"id": null,
"name": {
"value": "Get some eggs",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "Get some Milk",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "Take out the trash",
"synonyms": []
}
}
]
},
{
"name": "timer_time",
"values": [
{
"id": null,
"name": {
"value": "12",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "30",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "50",
"synonyms": []
}
},
{
"id": null,
"name": {
"value": "2",
"synonyms": []
}
}
]
}
],
"intents": [
{
"name": "addnote",
"samples": [
"Take a note to {task}",
"Remind me to {task}",
"Add a note to {task}"
],
"slots": [
{
"name": "task",
"type": "task_type"
}
]
},
{
"name": "AMAZON.CancelIntent",
"samples": []
},
{
"name": "AMAZON.HelpIntent",
"samples": []
},
{
"name": "AMAZON.StopIntent",
"samples": []
},
{
"name": "deletenotes",
"samples": [
"Delete Note",
"Clear all notes",
"Delete all notes",
"Remove all notes"
],
"slots": []
},
{
"name": "metric",
"samples": [
"Convert {value} {from_metric} to {to_metric}",
"{value} {from_metric} to {to_metric}",
"how much is {value} {from_metric} in {to_metric}"
],
"slots": [
{
"name": "value",
"type": "metric_value_type"
},
{
"name": "from_metric",
"type": "from_metric_type"
},
{
"name": "to_metric",
"type": "from_metric_type"
}
]
},
{
"name": "recipes",
"samples": [
"Show me the recipe for {recipe_name}",
"Hot to make {recipe_name}",
"Recipe for {recipe_name}"
],
"slots": [
{
"name": "recipe_name",
"type": "name"
}
]
},
{
"name": "temp",
"samples": [
"Whats the cooking temperature",
"whats the temperature of the dish",
"whats the temperature "
],
"slots": []
},
{
"name": "timer",
"samples": [
"Set timer for {time} seconds",
"Create timer {time} seconds",
"Timer for {time} seconds"
],
"slots": [
{
"name": "time",
"type": "timer_time"
}
]
}
],
"invocationName": "kitchen assistant"
}
}
Lambda is a serverless compute platform provided by Amazon, it allows you to deploy server apps without going through the process of setting up a server.
Now that you have set up the interaction model it is time to set up the AWS lambda function. For this you will need to login to the AWS console page. Next, select AWS lambda and select create lambda function.
- Function type - select Blueprints
- From Blueprints - select Alexa facts skill
On the next page you will require to fill in, info of the lambda function and you should also be presented with some example code of an Alexa facts app. Fill in the details on the page as follows -
- Name - Select any name for your lambda function
- Role - Create a new role
Once you have created the lambda function, add an input trigger of Alexa skills kit and also enter the Alexa skill id generated in the previous step. Next, you need to replace the nodejs app code with the one below and hit save.
'use strict';
var https = require('https');
var http = require('http');
var server_url = "Your Ip Goes here" //Replace the value with the ip of your nodejs server
const Alexa = require('alexa-sdk');
const APP_ID = undefined;
function check_timer(){ //Function to check if timer is already running
var endpoint = server_url+"/readtime";
http.get(endpoint, function (res) {
res.on('data', function (body) {
body = JSON.parse(body);
return body.time;
});
}).on('error', function (err) {
console.log('Error - ' + err.message);
});
}
const handlers = {
'LaunchRequest': function () {
this.response.speak("Hi im your kitchen assistant, I can set reminders, take notes, show recipes and lot more.").listen("How may I help you today");
this.emit(':responseReady');
},
'addnote': function () { //handler function for adding notes to the display
var self= this;
var note = this.event.request.intent.slots.task.value;
var endpoint = server_url+"/addnote?q="+ encodeURI(note);
http.get(endpoint, function (res) {
res.on('data', function (body) {
//body = JSON.parse(body);
self.response.speak("Note "+note+" has been added.");
self.emit(':responseReady');
});
}).on('error', function (err) {
console.log('Error - ' + err.message);
self.response.speak("An error occured please try again in some time");
self.emit(':responseReady');
});
},
'deletenotes': function () { //handler function for deleting all the notes on the display
var self= this;
var endpoint = server_url+"/deletenotes";
http.get(endpoint, function (res) {
res.on('data', function (body) {
//body = JSON.parse(body);
self.response.speak("All notes have been deleted");
self.emit(':responseReady');
});
}).on('error', function (err) {
console.log('Error - ' + err.message);
self.response.speak("An error occured please try again in some time");
self.emit(':responseReady');
});
},
'timer': function () { //function to set timer in seconds
var time = this.event.request.intent.slots.time.value;
var self= this;
var endpoint = server_url+"/settime?q="+ encodeURI(time);
if(check_timer()){
http.get(endpoint, function (res) {
res.on('data', function (body) {
//body = JSON.parse(body);
self.response.speak("A timer for "+time+" seconds has been set");
self.emit(':responseReady');
});
}).on('error', function (err) {
console.log('Error - ' + err.message);
self.response.speak("An error occured please try again in some time");
self.emit(':responseReady');
});
}
else{
self.response.speak("A timer is already running");
self.emit(':responseReady');
}
},
'temp': function () { //function to read the temperature of the dishes
var tem = 0;
var self= this;
var endpoint = server_url+"/readtemp";
http.get(endpoint, function (res) {
res.on('data', function (body) {
body = JSON.parse(body);
tem = body.temp;
if(tem==0){
self.response.speak("I'm sorry could you please check the connection to the temperature sensor");
self.emit(':responseReady');
}
else{
self.response.speak("The temperature is "+tem+" degrees celsius");
self.emit(':responseReady');
}
});
}).on('error', function (err) {
console.log('Error - ' + err.message);
self.response.speak("An error occured please try again in some time");
self.emit(':responseReady');
});
},
'recipes': function () { //handler function to display a recipe
var name = this.event.request.intent.slots.recipe_name.value;
var self= this;
var endpoint = server_url+"/newrecp?q="+encodeURI(name);
http.get(endpoint, function (res) {
res.on('data', function (body) {
body = JSON.parse(body);
stat = body.status;
if(stat == 1){
self.response.speak("Starting Recipie for "+name);
self.emit(':responseReady');
}
else{
self.response.speak("Sorry I could not find that recipe");
self.emit(':responseReady');
}
});
}).on('error', function (err) {
console.log('Error - ' + err.message);
self.response.speak("An error occured please try again in some time");
self.emit(':responseReady');
});
},
'metric': function () { //function to convert one metric unit to another
var val = this.event.request.intent.slots.value.value;
var from_met = this.event.request.intent.slots.from_metric.value;
var to_met = this.event.request.intent.slots.to_metric.value;
var self= this;
var endpoint = server_url+"/newrecp?val="+encodeURI(val)+"&to="+encodeURI(to_met)+"&from="+encodeURI(from_metric);
http.get(endpoint, function (res) {
res.on('data', function (body) {
body = JSON.parse(body);
stat = body.status;
if(stat == 1){
self.response.speak(val+" "+from_met+" is "+body.data+" "+to_met);
self.emit(':responseReady');
}
else{
self.response.speak("Sorry an error just occured");
self.emit(':responseReady');
}
});
}).on('error', function (err) {
console.log('Error - ' + err.message);
self.response.speak("An error occured please try again in some time");
self.emit(':responseReady');
});
},
'AMAZON.HelpIntent': function () {
this.emit('You can say tell me to set reminders, take notes, show recipes and lot more.');
},
'AMAZON.CancelIntent': function () {
this.emit('Goodbye');
},
'AMAZON.StopIntent': function () {
this.emit('Goodbye');
},
};
exports.handler = function (event, context) {
const alexa = Alexa.handler(event, context);
alexa.APP_ID = APP_ID;
alexa.registerHandlers(handlers);
alexa.execute();
};
You need to replace your the line of code which says your_server_ip with the ip address of the nodejs server you created earlier.
Next, navigate back to the your skills settings and in the configuration tab, paste the generated ARN (Amazon Resource Number), which can be found in the lambda function page.
After creating the Alexa skill and the lambda function, you have successfully created your project you can test kitchen assistant, via the skill builder page or you can enable your skill from the Alexa up under "My Skills" .
If you do not own an alexa enabled device you can use echosim.io to simulate an amazon echo device for testing the project.
Now that the kitchen assistant is complete, now lets design an add on device which works great with the kitchen assistant, a temperature sensor device which measures the temperature of the dishes you cook to make a better cooking experience.
CircuitThe circuit is simple, we will use a esp8266 to measure the temperature of a restive temperature sensor and then log it on the nodejs server.
This circuit can be powered by a Li-ion battery and can be made portable, and you should be able to carry it around your kitchen just like you would do for the Kitchen assistant device.
The esp8266 of the temperature sensor also uses Micropython so to upload the code you will need to flash the Micropython firmware and open up a webrepl to upload the file.
The below code needs to be written into the boot.py file of the esp8266. Make sure you replace the IP address field with the IP address of your nodejs server.
# This file is executed on every boot (including wake-boot from deepsleep)
#import esp
#esp.osdebug(None)
import gc
import webrepl
webrepl.start()
gc.collect()
import machine
import math
import time
import urequests as requests
import ujson as json
server_url = "Your ip goes here" #enter the ip of your nodejs server here
adc = machine.ADC(0)
THERMISTORNOMINAL = 10000
TEMPERATURENOMINAL = 25
NUMSAMPLES =5
BCOEFFICIENT =3950
SERIESRESISTOR =10000
def calculate_temp(): #this function calculates the temperature from the analog values
reading = adc.read()
reading = (1023 / reading) - 1
reading = SERIESRESISTOR / reading
steinhart = reading / THERMISTORNOMINAL
steinhart = math.log(steinhart)
steinhart /= BCOEFFICIENT
steinhart += 1.0 / (TEMPERATURENOMINAL + 273.15)
steinhart = 1.0 / steinhart
steinhart -= 273.15
return steinhart
#this loop updates the temperature to the server every 5 seconds
while(1):
r = requests.get(server_url+"/settemp?q="+str(calculate_temp()))
time.sleep(5)
The code converts the analog reading from the analog pin of the esp8266 and then converts those analog reading in the form of temperature data. The temperature is calculated and logged to the nodejs server every 5 seconds.
Demo VideoHere is the the video of the Arduino Kitchen Assistant in action.
Comments