Weather Station & FPGA Device Talking via the IOTA Network

Updated for 2019 China-US Young Maker Competition

Preface

This is an example implementation of services and devices on the IOTA network using a weather station, Raspberry Pi, FPGA, and a garage door opener. The IOTA network is still a young protocol with a lot of room for interpretation on how to best utilize it. While there have been attempts to market data for financial gain, I believe the more practical application lies in the sale of IoT devices with an ASIC style POW chip to talk on the IOTA network effortlessly. In order to prove just that, this project will create a server to allow subscriptions of services, devices, and applets. The service will be the weather station, which will post weather updates on the IOTA network specified by the service. The service will also include a GET request to the service's MAM root state so a device can connect at any time. The device in this project will be a raspberry pi that will be programmed to listen to the weather service and close the garage door if it begins to rain. An applet will be a predefined service, device, program tuple that other users can take advantage of assuming they have said device.

As a side note, let me expound on why I believe this model to be the future of IoT devices on the IOTA network. As of the time of writing, $1 is the equivalent of ~2.8597 MIOTA as of writing. Notice MIOTA is a million iota tokens. Meanwhile, device access for a weather station is usually between 500 and 5, 000 iota. In order to make just $1 with the ambitious cost of 5, 000 IOTA, the owner of the device would have to sell their data to over 570 people. Also, consider the case for the buyer. Weather stations aren’t very useful to data scientists by themselves. Having to search and personally collect/buy the list of stations available would be too much a hassle. Also consider the current economic model: Weather Underground boasts more than 250, 000 weather stations, which helps them accurately assess global weather trends. OpenWeatherMap allows you access every city in the world's weather for less than $200 a month. It seems like either direction is not the best we can do.

Thus, the primary goal of this guide is to present a better economic model for the future of IoT and their interconnectivity. The fundamental change is, in this instance, to make the subscription to all weather stations fee-less. You can do this by breaking the system into two parts. First, use the iota network to manage the data, security and protocol layers for you. This will offload the need/cost for central servers to manage/process the data. Instead the company would manage the service that posts/maintains the state of the data-stream on the IOTA ledger. Second, the company would include an ASIC POW device onboard the IoT device. This would ensure the device can prepare the data to be added to the network, so that the service can then verify/upload it and many more without the need of many/strong CPUs.

The rest of this tutorial will explain the methodology used to provide said services and utilize them with devices with minimal impact on the central server.

Prerequisites

For this project a Unix system is required. I used my Macbook for the majority of this project and used VirtualBox with an Ubuntu build to program the FPGA. The WS-2902A Ambient Weather station or equivalent ambient weather station will be necessary to provide an example service. The client is built with a raspberry pi (I used a model B+), a Xilinx compatible FPGA device (I ordered the Arty S7-50T), and wireless garage opener compatible with your garage opener. Some wire, soldering iron, solder, flux paste, and a breadboard are also required. NOTE: If you do not want to shell out the extra $90 to make the raspberry pi do its own POW, extend the garage client with your own module. There are plenty of tutorials on how to let other servers do the POW for you, including your own server. To listen to the garage door status, I used this reed switch with great success.

Designing the FPGA requires a little more effort, but considering this is one of the most exciting parts of this project, it is well worth it. The hat for the spartan is actually pretty easy to create yourself. Download the Gerber files from here. You can drag-and-drop the Gerber folder at JLCPCB and get a set of 10 for $2! Pretty sweet deal. As for the components, I created an XML file with the list of parts necessary to buy in order to properly build the hat. You can import the xml file easily into mouser via this link.

Step 1 - Setting up the weather station and server

WEATHER STATION

The WS-2902A Ambient Weather station measures wind speed, wind direction, rainfall, outdoor temperature, humidity, solar radiation, and the UV index. Lucky for us, this station uploads its data to Ambient Weather and the company allows access via their own API. NOTE: the station does not come with a pipe to mount to, so you must provide this yourself, otherwise the install is pretty self explanatory. Here is how mine ended up:

During the setup, you need to create an account online to utilize the data. Do so here. If you have set up your device properly, you should see:

Lastly to finish setting up the weather station click the avatar at the top right. At the bottom, you need to create both an APP key and an API key. Afterwards, you should see something like so:

At this point, keep those keys somewhere, as you will use them with the server.

SERVER

The server is the fundamental component to saving information about the client, writing instructions to the client, managing weather station subscriptions, posting weather station(s) data to the ledger, and managing both services and applets. Think of the server as IFTTT without the UI, and to circumvent this, we will use commands saved in the repository to complete our future tasks.

So let's get started. First download the repository and the dependences

# clone the repo
git clone https://github.com/iftt/demo-backend.git
# install dependences
yarn # or npm install

Now create an.env with the following:

PORT=3001
JWT_SECRET=m7ugp0jKc4RoRJvXtnCJrETheIRZENay
JWT_EXPIRES=15d
WEATHER_API=00000000bec84a09aa12094c4b3eac071c733dbc80834fc5a8d02fe840000000
WEATHER_APP_API=000000005ac24e28a5d6f76bb4b710c4fa5c76f1129b4002905afe31c0000000
WEATHER_TANGLE_SEED=I9XZHZETFFDYRWISYGGONGSULUTERQXQYZRALTNXWXQMRIORUSTTCKJVYNHCBWKGSDVEKIBXMQMOJKPQBX 

You need to substitute some of the values with your own. The API values you just created with your ambient weather account. To create a secure seed run the following

# Linux
cat /dev/urandom |tr -dc A-Z9|head -c${1:-81}
# OS X
cat /dev/urandom |LC_ALL=C tr -dc 'A-Z9' | fold -w 81 | head -n 1

to create a JWT secret, we expand the keyspace and changed the length

# Linux
cat /dev/urandom |tr -dc a-zA-Z0-9|head -c${1:-32}
# OS X
cat /dev/urandom |LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1

Assuming you completed all the above, the server is ready to run. Do so via:

# run the server via yarn
yarn run server
# run the server via npm
npm run server

All the modules I built have debugging build in, so if you run into any errors or want to see what's happening under the hood:

DEBUG=* yarn run server

Now that the server is up and running, you can test that it will indeed listen to commands. Inside apiTest.js ensure that TEST is uncommented and run the file.

Upon success, let's add your weather station to the AmbientWeather model so that we start listening for data. Comment out the TEST code and uncomment SUBSCRIBE WEATHER STATION. Run the apiTest file again. Now every minute (or every 5 minutes depending on your station settings) you should start getting the MAM root posted to the console if it is working.

You can check the current root by uncommenting the GET NEXT ROOT code. At this point I recommend checking all of the available commands to get an idea of what you can search with the current API.

The last thing you should do before moving on is uncommenting CREATE A NEW SERVICE inside the apiTest file and also at the top, uncommenting the weatherService JSON file. This will spit out the resultant weather service ID. You will use this id to program the device later on.

Great, your server is ready for the garage device!

WHAT'S HAPPENING UNDER THE HOOD

There are essentially two primary pieces in the demo-server, the weather service, and the server hosting service and device data. To ease the setup complexity, I attached the weather service to the server, but as you did earlier, you still have to create the service so the server knows of its existence. When you subscribe your weather station to the server, the weather service module will listen to the ambient weather api and publish the weather report to the tangle. Meanwhile, after every publish, the MAM root is updated and this is recorded for posterity. When you request to create a service, A data store manager called NeDBrecords the data format and assigns a key for future lookups. Although we will go into greater detail later, creating, getting, and programming a device is done in similar manner.

Now, while the iota network manages data security and the communication layer for us, one very crucial module for this project was creating an efficient Tryte communication layer. Consider the weather stations large messages. How could we bundle thousands of these weather station messages to both be memory efficient and allow other devices to decode and understand them?

First to manage encoding, I created a module that properly encodes and decodes basic types like numbers, strings, dates, boolean, arrays, etc. to the fewest Trytes possible while also being compatible with low level languages to support encoding/decoding on embedded devices. Luckily IOTA has also created an efficient geocoder that worked well with the module.

Second, we manage encoding size with the aforementioned tryte-buffer module encodes/decodes full objects into Trytes and back. The weather station is a great example of why this is useful. Let's say for instance 5, 000 weather stations are subscribed to our weather service. One message alone via the tryte-buffer module would take up roughly 108 characters, but converting the JSON to a string and then subsequent Trytes would take up 904 characters. Since Tryte messages have a 2187 character limit, the total number of messages each would take up would be around 247 and 2, 067 respectively. While this module does indeed add a layer of complexity, I believe it not only dramatically improves total message saturation on the tangle but ensures uniformity of data, so that embedded devices can then interpret easier.

Step 2 - Set up the FPGA device

PREFACE

I want to give a quick shoutout to Thomas Pototschnig. Not only is his work creating the FGPA POW device really impressive, but this project would have only been an idea rather than a demo if he hadn't put the hard work in to prove the feasibility of POW for embedded IoT. You can see more of his work here, here, and here.

SOLDER THE HAT

Although Thomas Pototschnig updated the FPGA project significantly, we will be using the revision I was able to get my hands on in time for this project.

This is a very involved and perhaps intimidating process for those of you who have not soldered on a PCB board before, much less SMD components. I had a lot of luck with flux paste, it works like glue to keep the SMD in place while you solder, and helps ensure the board itself isn't burned in the process.

Reiterating from the prerequisites, to build the hat, download the Gerber files from here. You can drag-and-drop the Gerber folder at JLCPCB and get a set of 10 for $2! Pretty sweet deal. As for the components, I created an XML file with the list of parts necessary to buy in order to properly build the hat. You can import the xml file easily into mouser via this link. The parts of the hat you actually need to build are shown in the following photo:

NOTE: I am going to tell you to also ignore J7. It is used for debugging purposes and is really not necessary to build the hat. However, the parts I gave in the mouser XML file will allow to you add it if you desire.

At first I thought soldering such small SMD pieces would be tough, but reading some blogs and watching a few videos, I completed the setup/soldering/cleanup in about half an hour. The final result looks as follows:

PROGRAM THE DEVICE

This is the most involved part of the process and takes some time. Explained in the prerequisites, I used Ubuntu via a VirtualBox to install Xilinx and program the device. Just incase make sure git, unzip, and build essential are installed:

sudo apt-get install build-essential git unzip

Once you're within a Linux device, install Xilinx (Be sure to get version 2018-2, as each version is not compatible with the last). The install link is here, but it is recommended to use the self installer here.

To run the installation, navigate to Downloads and run the bin file like so:

sudo ./Xilinx_Vivado_SDK_Web_2018.2_0614_1954_Lin64.bin

This will launch the installer. During the process, be sure to choose the webpack option:

This is where you will probably need to take coffee brake, at best you are looking at a wait time of about an hour.

If you did not install with sudo (and even sometimes if you did) you may need to run the usb Vivado driver install (Skip this step and come back if the hardware manager can't find the device):

cd VIVADO_INSTALL_LOCATION/Xilinx/Vivado/2018.2/data/xicom/cable_drivers/lin64/install_script/install_drivers
  
sudo ./install_drivers

If you succeeded, open Xilinx and open the hardware manager:

This is the trickiest part. First, If you are using a virtual desktop, you need to open up the USB ports:

Next, take a look at the following photo to understand how to let the virtual Ubuntu see the device:

At this point in the hardware manager you should see "No hardware target is open", next to that click "open target->auto connect"

Great, you should be connected now. You should be seeing something like this:

Now, the FPGA itself is only SRAM based, so it loses its configuration with every power-cycle. That means even if you program the device, after a reboot it's going to forget what you taught it. To flash the device permanently you have to add an SPI flash device in the hardware manager of Vivado. Richt-click on the Digelent device xc7s50_0 and select "Add configuration memory device" and select the SPI flash shown in this image:

You're going to see the SPI memory is ready for the program file, but we are not quite ready. At this point, you first want to program the FPGA with the bit file given here. First right click the xc7s50_0 again and this time select "Program Device". Navigate to the bit file and run. The device should have two green lights always on. One near the red PROG button and one on LD2. If that is the case, you successfully programmed the device. Now we need to store that data in the ram. In the Vivado toolbar navigate to tools->Generate Memory Configuration File... and use the configuration shown here to create the bin file:

Upon clicking ok and creating the bin file, your last step is to right click the the SPI memory and select Program Configuration Memory Device. Select the bit file you created and click ok:

Congratulations! You have a working FPGA device! You can communicate over the serial port using the API provided.

Step 3 - Creating a garage door client with the FPGA and Raspberry Pi

INSTALL

Here is the last piece of the puzzle. If you've made it this far, the rest is a breeze. Let's start with setting up the raspberry pi. Since the device will be headless, download a Raspbian stretch lite image. To write the image to a microSD card, I used Belena's Etcher.

Upon writing the IMG file, etcher should auto unmount the image, but you are not done, do not eject the card just yet. Navigate to the images boot folder with a terminal. First create an empty file called ssh. That will enable ssh on the device. Next, if using wireless, we need to configure it. Still in the boot folder, create a file wpa_supplicant.conf with the following configuration:

network={
    ssid="YOUR_WIFI_SSID"
    psk="YOUR_WIFI_PASSWORD"
    key_mgmt=WPA-PSK
}

Your image is ready now. Plug it into the device and boot up the raspberry. To configure your Pi, you need the IP address. You can find this in your Router’s DHCP lease allocation table. Once you know your raspberry pi's local ip, we need to ssh in:

ssh pi@192.168.49
Enter Password: raspberry 

Be sure to change the password right away:

passwd

Next, update, upgrade and prep

sudo apt-get update && sudo apt-get upgrade
sudo apt-get install build-essential git unzip
sudo apt-get install -y nodejs

Now let's install the garage client:

git clone https://github.com/iftt/garage-client.git
cd garage-client
npm install

WIREUP THE REED SWITCH AND GARAGE OPENER

For this project, I used this reed switch and a wireless transmitter that came with the garage door opener.

The blue pin and 1k resistor both wire directly to the reed switch. In my case the reed switch was in a "closed" state when the garage was open, so the GPIO pin would update a 1 when the garage was open. The green pin in my case actually went to a 1K resistor before connecting directly to the garage transmitter. I also grounded the transmitter to the GND rail on the breadboard, as supplying extra current to a battery device would be a bad idea without an escape route. Here is what my transmitter looks like:

It took a little time to figure out how the transmitter worked. Does the "key" sent to the opener get transmitted through the switch when it's closed or does the switch simply supply 3.3V to a pin that activates the transmitter? I figured since push buttons have debounces it had to be the latter. Turns out I was right and supplying 3/4 of a second of voltage to the pin worked just fine. I personally prefer to use a Saleae logic anaylizer for reverse engineering these kinds of devices.

Once the switches are connected, you are ready for the next step.

TEST SWITCHES, CLIENT, & PROGRAM THE DEVICE

Inside the garage-client repository you downloaded, you can test the connected devices and switch. First, the garage transmitter can be tested with:

node doorOpenerTest.js

You can verify the raspberry pi is talking to the FPGA device correctly as follows:

node fpgaTest.js

Lastly, you can test the reed switch by opening and closing the garage door as explained above and reading the output from

node garageDoorStatusTest.js

If all these devices are working as intended, you are ready to setup the environment variables and run the client!

In the root directory of garage-client, create a file called .env and add the following

API_TOKEN=1234567890
CLIENT_SECRET=TEST9SECRET
CLIENT_TANGLE_SEED=0000JYXUUZVHMCDLNEFA9ZBRM00000PJFLEPMNEFWXOHCTVZWBLCLE9HUOLYLWS9NMJDOKQMOKINXQA
DEVICE_ID=477a5971-3e9d-4eae-98fe-22cb4c153dc1
SERVER=192.168.0.46:3001
GARAGE_OPEN_STATE=1

The CLIENT_TANGLE_SEED is created the same way you did for the server

# Linux
cat /dev/urandom |tr -dc A-Z9|head -c${1:-81}
# OS X
cat /dev/urandom |LC_ALL=C tr -dc 'A-Z9' | fold -w 81 | head -n 1

For the DEVICE_ID, I used a UUID-V4. You can create a random UUID in the terminal using the command:

uuidgen

The SERVER obviously just points to the address and port of the currently running demo-server. Fix accordingly.

The GARAGE_OPEN_STATE lets the code interpret what the value coming from the reed switch means. For instance, in my case when the garage door was open, the GPIO pin was reading a 1. Therefore I wrote down a 1. However, if your switch is placed elsewhere and your GPIO pin reads 0 while the garage is open, adjust.

Now you should be ready to connect to the client.

node runGarageClient.js
# OR run in debug mode
DEBUG=* node runGarageClient.js

I recommend the first time running the program in debug mode to ensure all the modules are doing their job, although you will at least see "FPGA device is ready" if the FPGA device is working.

One last step while the client is running. We want to build an applet to show the power of a device listening to the weather service. I pre made the applet for you in the test folder entitled weatherProgram.json. Although you can program the device via the server (and kind of the point of the project), to make things easier for you I setup a program inside the client folder to run:

node programTest.js

Be sure to check the contents of the weatherProgram.json as it is quite easy to understand what is happening. The client will start listening to the MAM messages provided by the weather service. The program itself will listen for the weather to start raining and trigger the action. The action checks if the garage door is open, and close said door if true.

If you don't want to wait for actual rain to come to ensure it's working (I don't blame you) The demo-server repository came with a fake server that emulates data that claims it started to rain. If the garage door is open, it will close. So stop the current server and run:

node testServer.js

You will obviously have to restart the client as well, since the MAM root message will have changed.

SO WHAT'S HAPPENING UNDER THE HOOD?

The garage-client can be broken down to 5 primary functions.

First the garage client manages current and future connections via websockets. Setting the environment variable ORIGIN for security is crucial in a production scheme, and this is why. Limiting to the local network and the server would make the most sense. In order for the server to find the device, the client pings the server every 15 minutes. This is done by the next primary function.

The service manager manages any incoming and outgoing traffic involving tangle messages. When programming the garage client, we need to know how the service we are listening to works and what the messages look like. When we do add a service, we create a program to handle the specific data and instruction set. The instructions are defined using the cleverly made JsonLogic module. In the case of the weather program we will be using, if the eventrainin variable changes from 0 to any value (its started to rain for the first time in a while) the logic triggers the weather program to emit an action. Again, the weather program we are using specifies to open the garage. if the garage state changes, the next primary function kicks in.

The third primary function is the FGPA device and how it publishes messages to the tangle. The icc-fgpa-protcol is quite simple thanks to the clever apiThomas created. The fpga uses a serial port to communicate, and essentially, all this module has to do is properly send the attachToTangle message to the device and let it take care of the rest. upon completion, we send the result back to the garage client who in turn sends it to an IRI server. To ensure we don't overload the the device and also because the MAM module does not have a built in queue, I used one provided on NPM.

The last two primary functions are the reed switch and the transmitter to open/close the garage door. There is a raspberry pi GPIO module that makes handling these very easily. We use the reed switch to watch the status of the garage door and the transmitter to open/close the garage door when instructed to do so.

Done

That's it, at this point my hope is you should be able to make your own services, devices, and applets. The goal was to give you, dear reader, the tools and modular code necessary to build out your own IOTA projects quickly and efficiently, and I hope I have succeeded.

Improvements

I wish I had more time on this project as I had a lot of fun building it. I would like to talk a little about how the module could be improved and some thoughts about the direction of IOTA.

CLIENT

Currently, the raspberry pi client is using a few second intervals to check if the MaM root has changed, which is less than ideal. I would love to see the MAM module include ZMQ support, and I wish I had implemented it myself in this project. This would have a powerful impact on performance if the device was perhaps listening to several services.

The biggest problem I faced when deciding the direction to take for the client was whether to add IoT compatible IRI like Hercules or maybe the recent even lighter node ICT. My gut tells me ICT will eventually become a viable option for production. I ended up deciding to include neither for the demo, both given the time it would take to implement either and ICT's current lack of mainnet / MaM support. [I think ICT does support MaM now, but a little late for this project]

SERVER

The server is just a demo, but I think it's enough proof of concept that building out a user interface would not be so tough. Within the UI, it would be great to see Websockets implemented to talk to the client directly and make subtle changes. This would perhaps be another way of programming the client as well. The other missing component I would have liked to have added to the demo is support for applets. These would be predefined instruction sets for specific services and devices.

Final Thoughts

That's it. I appreciate you reading. I put a lot of time and thought into what would best express the value of the IOTA Tangle, and I hope you now see my perspective on IoT's relationship with it. If you have any ideas or suggestions about this project, feel free to contact me. My GitHub handle is @CraigglesO.