Sensors Connectivity

Why Decentralized Sensors Network?

There are public monitoring networks such as German project sensor.community, but they use the usual client-server architecture, which in this case is a drawback. Data from all sensors along with user requests all are sent to one server, which cannot always handle such load. So there are situations when the map with data is not available at the most important moments.

With Robonomics Network, sensors send data to several different servers, and any user can bring up the Sensors Connectivity server for their sensor and see it on the map. The map itself is not overloaded because it is a decentralized application (dApp) that works directly from your browser with the data that the servers send to the IPFS pub-sub channel.

About Sensors Connectivity

Decentralized Sensors Network uses the sensors-connectivity Python module (source code). This module allows any user to launch their own server to receive data from sensors and process it further. At the moment, the developers have launched several such servers and any sensor can send data to them. Running multiple servers avoids data loss in case of problems with one of them, as the sensors will switch from a non-working server to a working one. Basically, you can think of the Sensors Connectivity module as a black box with one input (sensor data) and many outputs.

Sensors Connectivity module is a set of stations (station_1, station_2... station_n), which receive various data, including data from sensors via HTTP protocol. Also it can be sensors connected to the computer via USB or any other data source. Data received from the stations are processed by the module and passed to feeders (feeder_1, feeder_2... feeder_n). The feeders send the processed data to the user; in our case the data is sent to the decentralized IPFS channel.

A map of Decentralized Sensors Network is a decentralized application (dapp) running on the computer. It reads data from the IPFS channel and displays it on the map. There is no direct connection between the server that collects data from the sensors and the map that the user sees; data is transferred between them via IPFS pubsub, which reduces the load on the servers.

In addition, from time to time, a file with data for the last period of time is stored in IPFS, and a hash of this data is then recorded on the blockchain. Since IPFS is a content-addressed network, storing data in it ensures that any data change does not go unnoticed, because the address of the needed file contains a hash of its content, which will change with any data change. The blockchain is used to pass the hash to the user, who can use it to get the needed data from IPFS (this happens when you request a history of the map).

Sensor Connectivity Setup for Linux

Pre-requirements and Installation

To build sensors-connectivity module the IPFS daemon has to be installed with version no greater than 0.8.x. Assuming, you work on Linux, execute the following (for version 0.8.0):

1. To build sensors-connectivity module the IPFS daemon has to be installed with version no greater than 0.8.x. Assuming, you work on Linux, execute the following (for version 0.8.0):

wget https://dist.ipfs.io/go-ipfs/v0.8.0/go-ipfs_v0.8.0_linux-amd64.tar.gz
tar -xzf go-ipfs_v0.8.0_linux-amd64.tar.gz
cd go-ipfs
sudo bash install.sh
ipfs init

2. Install package with development tools python3-dev and package installer for Python pip:

sudo apt install python3-dev python3-pip

3. Install the module as a PyPI package:

pip3 install sensors-connectivity

If you see following warning:

WARNING: The script sensors_connectivity is installed in '/home/test2/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

Run next command:

cd ~
export PATH="/usr/local/bin:$PATH"
source .profile

Configuration

1. Create directory for configuration file and database file wherever you want. This database will save IPFS hashes of sensor data, timestamp and service status:

cd ~
mkdir sensors_connectivity
cd sensors_connectivity
touch database.db

Name of the database file can be any, but extension must be.db

2. Create configuration file in the same directory:

touch my_config.json

3. Copypaste the following to the file and insert full path to the database file in db_path field. This configuration will be enough to get sensors data over HTTP and send it to the Robonomics map:

{
"general": {
"publish_interval": 30,
"db_path": "/PATH/TO/DATABASE>"
},
"comstation": {
"enable": false,
"port": "/dev//PATH/TO/BOARD>",
"work_period": 300,
"geo": "00.000000,00.000000",
"public_key": ""
},
"httpstation": {
"enable": true,
"port": 8001
},
"mqttstation": {
"enable": false,
"host": "localhost",
"port": 1883,
"topic": "/freertos_mqtt_robonomics_example/#",
"username": "",
"password": ""
},
"robonomics": {
"enable": true,
"ipfs_provider": "/ip4/127.0.0.1/tcp/5001/http",
"ipfs_topic": "airalab.lighthouse.5.robonomics.eth"
},
"datalog": {
"enable": false,
"suri": "",
"dump_interval": 60,
"temporal_username": "",
"temporal_password": "",
"pinata_api": "",
"pinata_secret": ""
},
"dev": {
"sentry": ""
},
"frontier": {
"enable": false,
"suri": ""
},
"trackagro": {
"enable": false,
"token": ""
}
}

4. Optional: Publish Sensors Data with Robonomics Datalog

To upload your sensor's data to with datalog function Robonomics parachain. Datalog is analog of telemetry in web3 technologies. The function is meant to create a sensor's data snapshot each period of time, which increase reliability of data.

You have to have XRT tokens on your account.

Insert into the configuration file:

1. a private key from Robonomics parachain account to suri field

2. a period of time for collecting log in seconds to dump_interval field

3. (optional) credentials to upload files to Temporal.Cloud in temporal_username, temporal_password fields

4. (optional) credentials to upload files to Pinata in pinata_api, pinata_secret fields

Obtain IPFS ID

Get IPFS ID of your module by running:

$ ipfs id
{
"ID": "QmUZxw8jsRpSx5rWkTpJokPGKvWihTrt5rbRCFXzJ4eCAP",
"PublicKey": "CAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC/uMV3rLM/C+LOh2DGPo3chr+VM+vyYMKi...
...

We will require it for configuring the Map.

Launch

1. Launch IPFS daemon:

ipfs daemon --enable-pubsub-experiment

2. After config is set, run the service with the path to config file in another terminal:

sensors_connectivity "path/to/your/config/file”

3. You will see logs in the terminal (also, they will be added to ~/.logs). Example:

sensors_connectivity test.json
2022-09-02 14:08:48,408 - INFO - Getting data from the stations...
2022-09-02 14:08:48,409 - INFO - airalab-http-v0.8.0: [[]]
2022-09-02 14:08:48,409 - INFO - Sending result to the feeders...
2022-09-02 14:08:48,411 - INFO - Checking data base...
2022-09-02 14:09:18,410 - INFO - Sending result to the feeders...
2022-09-02 14:09:18,410 - INFO - Getting data from the stations...
2022-09-02 14:09:18,411 - INFO - airalab-http-v0.8.0: [[]]
2022-09-02 14:09:48,411 - INFO - Sending result to the feeders...
2022-09-02 14:09:48,412 - INFO - Getting data from the stations...
2022-09-02 14:09:48,413 - INFO - airalab-http-v0.8.0: [[]]
2022-09-02 14:10:18,413 - INFO - Sending result to the feeders...
2022-09-02 14:10:18,413 - INFO - Getting data from the stations...

Hardware setup

To send data to your custom server you need to set up a custom API in the sensor's configuration. To do this, find an IP of your sensor and open it in a browser. Then, navigate to Configuration and in the tab APIs click on Send to own API and specify the server address and port from the Sensor Connectivity configuration file.

Robonomics Sensors Measure Analytics and Archive Node

Robonomics Sensors Measure Analytics and Archive Node or RoSeMAN is a service for accumulating sensors data to show measurement history.

Requirements

RoSeMAN requires MongoDB database server, it is assumed that you already have it. Also, you have to turn on datalog option for Sensors Connectivity module, as shown in Configuration part #4. You should have free XRT tokens on your Robonomics account, which connected to Sensors Connectivity module.

Setup

1. Download repository:

git clone https://github.com/airalab/RoSeMAN.git
cd RoSeMAN

2. Create configuration files:

cp config.template.json config.json
cp agents.template.json agents.json

3. Open config.json file and edit database path:

...
"DB": {
"path": "mongodb://localhost:27017/rosemandb"
},
...

4. Add public address of your Robonomics account to agents.json file. You can add several addresses to file, if you want to collect data from different Sensors Connectivity modules.

5. Install dependencies and build server:

yarn install
yarn build

6. Start RoSeMAN server:

yarn start

Web server should launch at http://127.0.0.1:3000.

Post-installation

After deployment of the RoSeMAN to a server you have to get public IP address or URL to server.

Sensor Map Deployment

After assembling a sensor and setting up Sensors Connectivity module it is time to deploy personal decentralized sensor map. To do it using GitHub Pages, start by forking the repository and clone it. To enable GitHub Actions, go to the repository page, click on the Actions tab, and set up a workflow if you haven't done so already. To successfully deploy the map, you will need to modify certain files:

Option 1: you don't have Domain

1. In .github/workflows/main.yaml remove cname: sensors.robonomics.network.

2. Add the following code right below runs-on: ubuntu-latest in .github/workflows/main.yaml:

permissions:
contents: write

3. In vite.config.js add base: "/<repository name>/", to the defineConfig object just above the plugins section, replacing <repository_name> with the name of your fork (default will be sensors.robonomics.network)

Option 2: you have Domanin

1. In .github/workflows/main.yaml change cname: sensors.robonomics.network to your CNAME.

2. Add the following code right below runs-on: ubuntu-latest in .github/workflows/main.yaml

permissions:
contents: write

Map Configuration

1. In src/config.js insert your RoSeMAN URL into REMOTE_PROVIDER field:

...
},
REMOTE_PROVIDER: "https://your.roseman.example.org/",
WIND_PROVIDER: "",
MAP: {
...

2. In src/agents.json add IPFS ID of your Sensor Connectivity module. If you wish to display only the sensors from your Sensor Connectivity module on the map, remove all other IDs except yours.

Manage Github Pages

After making modifications to the files, you can deploy your instance of the map by following these steps:

1. Commit and push the changes to your forked repository.

2. Wait until the actions have successfully completed.

3. Go to the Pages section of your repository Settings.

4. Enable GitHub Pages by selecting Deploy from a branch as the source

5. Choose the gh-pages branch and the root folder.

6. Save the settings, and GitHub Pages will deploy your instance of the map.

7. Access it using the provided GitHub Pages URL.

If you don't see the GitHub Pages URL try to reload the page.