CRowdsourced Open Weather System (CROWS)

Weather forecasting is arguably one of the first human activities. It is imbued with the strong praticality of hunters and gatherers and the urge of understanding the world we live in.

While in ancient times forecasting was based on very simple signs and omens, nowadays it exploits huge amounts of data fed to complex physics models.

Commercial weather services, like OpenWeather, collect data from tens of thousands of connected weather stations every minute. This data is then enjoyable by commercial entities or enthusiasts via API in a freemium price model.

However, no revenue stream is in place for weather station owners. In an age where content creation is becoming more and more a gold mine, why don't weather station owners get paid for their data?

This project will use the IOTA protocol and token to sketch a Proof-of-Concept for a crowdsourced open weather system (CROWS), where owners get paid for quality data collection and creation.

Project idea

The PoC includes two main components.

A Weather Station aka Client that handles the weather station hardware, takes and posts measurements.

A Server that handles data ingestion, manages registered stations and makes payments to station owners.

There are a few requirements:

• the system should work as close as possible to existing weather collection systems with the additional layer that allows payments.
• the client should be able to make lots of provable transactions for little or no fees
• the server should be able to make lots of very small payments for little or no fees

For these reasons the communication formats are based on an OpenWeather-like protocol. The IOTA protocol, instead, will be the perfect match for multiple provable and free transactions, either for data or for micro payments.

Scenario

Not having a weather station at hand can be critical while developing a project for a weather station system. But hey... since we are speaking about a PoC we can safely mock our weather station. We chose a simple RaspberryPI 2 model B with an attached DHT22 sensor and an Edimax EW-7811Un WiFi adapter.

The server on the other hand should be a scalable piece of infrastructure. In our case we are using a simple desktop computer, while in reality we would seek a proper cloud infrastructure.

Sadly most of the "fun" of this project is behind the scene: communication, network and payment protocols,... So we added a simple web interface on the server to show what is going on in addition to the console output. The web interface theme is based on AdminLTE.

Both desktop PC and PI are capable enough to run Node.js, so both are running their own application. The code is available in a multirepo on GitHub. The code is written in ES6 Javascript and runs on Node.js >10.

Both actors have a IOTA wallet to send transactions to the IOTA DevNet Tangle. The IOTA token can be provisioned via the handy IOTA DevNet faucet.

How does it work?

StationRegistration

The Client wakes up and tries to register itself to the data collection Server. This includes passing info such as the station name and geolocation.

In OpenWeather this is achieved via a JSON REST API. CROWS uses the same data format but has an additional field payment_address, so that the server may store it as the IOTA address for future payments.

As response, the client obtains a station_id used for future communications.

This exchange uses a standard REST API (on top of Express) as it is probably more suited to a request-response paradigm. Here is the UML representation of the protocol.

WeatherMeasurement

On a fixed schedule the weather station posts weather measurements such as temperature, pressure, clouds, rain, etc depending on the sensors available at site. The data is encoded in a JSON format exactly similar to the OpenWeather one, but the IOTA Tangle is used as network transport. Each measurement is posted as zero-value transaction to a known server address.

On "the other side of the Tangle", the server uses IOTA ZeroMQ, a streaming protocol for asynchronously listening of transactions to that address. Every time a transaction is received, the measurement data are decoded and stored for future use. Additionally, a counter for a unpaid_measurements is incremented for the station of origin.

Due to how the IOTA consensus works right now, it takes about 1-2 min for a transaction to be confirmed by the DevNet Tangle. In this timeframe a Coordinator takes valid transactions and packs them in a "milestone". At this point, the transactions are confirmed and ready to use.

Payment

As time goes by, stations post huge amount of measurements. The server periodically runs a payer process. It checks each station unpaid measurement counter: if it is above a certain threshold, it pays via IOTA some tokens to the station payment_address (in this case as many tokens as unpaid_measurements).

How does it run?

Server

1. Clone the repository and prepare the dependencies

git clone https://github.com/thundo/crows-iota.git      
cd core      
yarn install      
cd ../server      
yarn install      

2. Generate a IOTA seed for your wallet. If you don't know how to do that, follow these instructions

3. Create your config/local.json file which overrides the default configuration. It should be something like

{      
    "iota": {      
        "seed": "P9LSRMEPXTBRFQTDVVVIRNEPCSJYNYAKWHZUIRLARWWLXMVOHHJZJOWPAKCVFRVCTWQGOCIBY9ZENDPUR"      
    }      
}      

The iota.seed should be the IOTA seed generated at point 2

4. (Optional) You can mess with the CROWS settings like payment interval, logging level or IOTA remote node and security. Just look at config/default.json and override the relevant parts in config/local.json

5. Start the server with

yarn dev      

6. During startup please note the address generated from your IOTA seed which will be the address the stations sends their measurements to

The same can be seen in web interface hovering above the QR code. By default the web interface is reachable at http://localhost:8080 once the server is running.

Raspberry Setup (Weather station with DHT22)

Install the Node.Js interpreter and the yarn package manager

Login via SSH to your PI then

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -      
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list      
curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -      
      
sudo apt install -y nodejs yarn git      

Client

1. Still logged on the PI, clone the repository and prepare the dependencies

git clone https://github.com/thundo/crows-iota.git      
cd core      
yarn install      
cd ../client      
yarn install      

2. Generate a IOTA seed for your wallet. If you don't know how to do that, follow these instructions

3. Create your config/local.json file which overrides the default configuration. It should be something like

{      
    "id": "CROWS_STATION_0",      
    "name": "Crows Weather Station 0",      
    "latitude": 45,      
    "longitude": 9,      
    "altitude": 80,      
    "iota": {      
        "seed": "OEJDD9YRHZLIPHCMTVMLREPISX9CQERNEXNCIXQKMQOLILXPIBMRGZPWWYXVWNQBHOZOBVVFRGOSJXS9M",      
        "serverAddress": "VTMPVYGXMIMJHYROFIHYBODLIXZYMAUTOWVEURKSDSLEHPENKSCWXVJJWIJPBHVAZNFHIDNVTUBIYMCB9"      
    },      
    "crows": {      
        "baseUrl": "http://192.168.0.1:8080",      
        "sensor": {      
            "pin": 4,      
            "type": 22      
         }      
    }      
}      

Let's review these settings:

id: an identifier for your station (string)

name: a display name for your station (string)

latitude, longitude, altitude: geolocation of the station (numeric)

iota.seed: the IOTA seed generated at point 2 (string). It should be the one generated at point 2

iota.serverAddress: the IOTA server address shown during server startup (string)

crows.baseUrl: the web address of the CROWS Server. By default it listens on port 8080 (string)

crows.sensor.pin, crows.sensor.type: configuration for the DHT sensor. Pin specifies the DATA pin connected to the sensor, while type should be 22 (or 11) depending on the DHT sensor type (numeric)

4. (Optional) You can override additional settings like logging level or IOTA remote node and security. Just look at config/default.json and override the relevant parts in config/local.json

5. Start the client with

yarn dev      

After a few moments the station should register and send measurements.

Limitations

As stated, this project describes a PoC. A lot of issues were overlooked both in terms of economics and technology.

On the economics side there should be a model able to support enough revenue streams for the stations and for the company running the service. If not sustainable this could jeopardize the feasibility of the system.

On the technology side many simplifications were adopted. For example, there is no persistence: the weather data and the registered stations are only stored in-memory. If the server goes down or restarts the whole network needs to be re-registered. The IOTA protocol does not take care of snapshots or fully support not reusing addresses.

In addition, the server sytems should be properly sized to handle the load of tens of thousands of stations. Similarly, the system running the stations should be powerful enough to run the Proof-of-Work part of the IOTA protocol.

The server wallet handling payments should always be filled with enough tokens, otherwise the server will be unable to make transactions.

All in all, these tech issues are minor and solvable.

A closing note on abusing the system. Having incentives for stations to post data can encourage system abuse. There should be processes that weed out stations flooding data or posting incorrect measurements.

Remarks

OpenWeather is a trademark of Openweather Ltd.

AdminLTE is copyright © 2017 AdminLTE.IO.

Hope you like this project as much as I did working on it. Find me on twitter @thundo. Cheers!