Spider Sense

Abstract

We are Marco Costa, Giuseppe Capaldi, Artem Savchuck from the Master Course of Engineering in Computer Science at "La Sapienza" University of Rome. This is a project made during IoT course.

The idea

Our idea was to use an ultrasonic sensor attached to a servo motor to make possible an obstacle detection at 180 degrees and to implement from that an application capable of triggering and broadcasting an alarm to the users.

This was thought to be applicable in all the situations in which you cannot see what's happening in a certain area. Suppose you are a private guard who has to keep watch without someone "watching your back." Or suppose you want to have a way to see in your smartphone how much your car is near to the wall when you're parking in your garage. Or even imagine a blind person that with a simple buzzer replacing radar animation or through vibration, can understand to be in front of an obstacle.

From this idea it was born "SpiderSense" project, recalling the famous power of Spiderman to understand if he's in danger like a sixth sense.

So in the next lines you will find a detailed documentation on the hardware we used and the software we made, along with the Github page, in which you can find all the code: Nucleo board, Android and Telegram bot codes.

External resources

• Github page: link
• Slides presentation: link

Hardware Components

STM32 NUCLEO-F401RE

The STM32 Nucleo board provides an affordable and flexible way for users to try out new ideas and build prototypes with any STM32 microcontroller line, choosing from the various combinations of performance, power consumption and features.

Board features:

• Two types of extension resources
• Arduino Uno Revision 3 connectivity
• STMicroelectronics Morpho extension pin headers for full access to all STM32 I/Os
• On-board ST-LINK/V2-1 debugger/programmer with SWD connector
• Selection-mode switch to use the kit as a standalone ST-LINK/V2-1
• Flexible board power supply
• USB VBUS or external source (3.3 V, 5 V, 7 - 12 V)
• Power management access point
• User LED (LD2)
• Two push buttons: USER and RESET
• USB re-enumeration capability: three different interfaces supported on USB
• Virtual Com port
• Mass storage (USB Disk drive) for drag'n'drop programming
• Debug port

Official website

STM32X-NUCLEO-IDB05A1

The X-NUCLEO-IDB05A1 is a Bluetooth Low Energy evaluation board based on the SPBTLE-RF BlueNRG-MS RF module.

Official website

Servo motor SG90

This is a common low cost servo motor, with many libraries for many boards like stm32 and atmega328p available in the open source community. It can rotate approximately 180 degrees (90 in each direction).

• Datasheet: link

Ultrasonic Sensor HC-SR04

HCSR04 sensor can measure distances from 2 cm to 400 cm with an accuracy around 1 cm. In the module you can find one Ultrasonic signal emitter, one Ultrasonic signal receiver and a control circuit. The sensor does not measure directly the distance, what you read on the digital signal is the measure of time passed in sound waves movement from the sensor to an obstacle and back to the sensor.

• Datasheet: link

Theory behind BLE connection

1 - BLE Protocols

BLE operates in the 2.4 GHz Industrial Scientific Medical (ISM) band and defines 40 radiofrequency (RF) channels with 2 MHz channel spacing. In BLE, when a device only needs to broadcast data, it transmits the data in advertising packets through the advertising channels. Any device that transmits advertising packets is called an advertiser. Devices that only aim at receiving data through the advertising channels are called scanners. Bidirectional data communication between two devices requires them to connect to each other. At the highest level of the core BLE stack, the GAP (Generic Access Profile) specifies device roles, modes and procedures for the discovery of devices and services, the management of connection establishment and security. The BLE GAP defines four roles with specific requirements on the underlying controller: Broadcaster, Observer, Peripheral and Central. The ATT (Attribute protocol) allows a device to expose certain pieces of data, known as "attributes", to another device. The ATT defines the communication between two devices playing the roles of server and client, respectively, on top of a dedicated L2CAP channel. The server maintains a set of attributes. An attribute is a data structure that stores the information managed by the GATT (Generic attribute profile). The client or server role is determined by the GATT, and is independent of the slave or master role (managed in the link layer). The GATT defines a framework that uses the ATT for the discovery of services, and the exchange of characteristics from one device to another. GATT specifies the structure of profiles. In BLE, all pieces of data that are being used by a profile or service are called "characteristics". A characteristic is a set of data which includes a value and properties.

2 - Profiles and services

The BLE protocol stack is used by the applications through its GAP and GATT profiles. The GAP profile is used to initialize the stack and setup the connection with other devices. The GATT profile is a way of specifying the transmission - sending and receiving - of short pieces of data known as "attributes" over a Bluetooth smart link. All current Low Energy application profiles are based on GATT. The GATT profile allows the creation of profiles and services within these application profiles. Here is a depiction of how the data services are setup in a typical GATT server:

The profile above has been created with three services:

• GAP Service, which is always mandatory to be setup

• Distance service

• Angle service

Each service consists of a set of characteristics which define the service and the type of data it provides as part of the service. Each characteristics contains details about the type of data and the value of the data. The characteristics are defined by "attributes" which define the value of that characteristic.

3 - BLE Operations

The above diagram describes the state machine during BLE operations. The operations are the following:

• Standby: Does not transmit or receive packets.

• Advertising: Broadcasts advertisements in advertising channels. The device is transmitting advertising channel packets and possibly listening to and responding to responses triggered by these advertising channel packets.

• Scanning: Looks for advertisers. The device is listening for advertising channel packets from devices that are advertising.

• Initiating: The device initiates connection to the advertiser and is listening for advertising channel packets from a specific device(s) and responding to these packets to initiate a connection with another device.

• Connection: Connection has been made and the device is transmitting or receiving.

− Initiator device will be in master role: it communicates with the device in the slave role, defines timings of transmissions.

− Advertiser device will be in slave role: it communicates with single device in master role.

Theory behind Telegram bot

1 - MongoDB

Our Bot uses MongoDB which is a cross-platform document-oriented database. By its nature, It's noSQL database which uses JSON-like documents with a schema.

Key advantages of MongoDB:

• First, it's very easy to install and use. Also, cloud-based services could be used such as mLab, which allow you bypass installation procedure and start immediately play with mongo;
• The very basic feature of MongoDB is that it is a schema-less database. No schema migrations anymore. Since MongoDB is schema-free, your code defines your schema;
• Extremely fast;
• Since, it is a noSQL database, then it is obviously secure because no SQL injection can be made.
• MongoDB supports, the search by regex and fields as well.

2 - Deployment on Heroku

We deployed our Bot to Heroku which is a cloud platform as a service (PaaS) supporting several programming languages (Ruby Java, Node.js, Scala, Clojure, Python, PHP, and Go).

Applications that are run on Heroku typically have a unique domain (typically "applicationname.herokuapp.com") used to route HTTP requests to the correct dyno. Dynos are isolated, virtualized Linux containers that are designed to execute code based on a user-specified command.

The platform provides free price category which allows you to use two dynos: one for receiving HTTP request and another for background job. For our task it is enough to use only the first one.

Heroku comes with useful and convenient CLI app, that allows you manage, deploy, run your app, configure dynos and much much more.

Project Overview

1 - Hardware Configuration

First of all it's necessary to mount the X-NUCLEO-IDB05A1 board on the NUCLEO-F401RE board. After this, it is possible to move on with the mounting of the sensor. Both the ultrasonic sensor and the servo are powered by the 5V of the board. Moreover, the servo is connected to D10 pin for setting the position, the ultrasonic is connected to D9 pin (Trig) to send a signal (high-frequency sound) and to D8 pin (Echo) to receive the signal. Finally the red led is connected to D2, while the green one is connected to D4. A simple plastic structure has been created to move the ultrasonic sensor along with the servo. The final result is shown below.

Schematics

Note: on top of Nucleo-F401RE board you would have the bluetooth module (IDB05A1) attached (not visible in the image), the pins remain the same

2 - Programming Nucleo board

The first thing to do is to include the needed libraries (Mbed, ultrasonic, servo, bluetooth, distance and angle services. The last two libraries has been written from scratch (see the github page for the code).

#include "mbed.h"
#include "HCSR04.h"
#include "Servo.h"
#include "ble/BLE.h"
#include "DistanceService.h"
#include "AngleService.h" 

After that, it's necessary to initialize the sensors passing the pins to which they was connected:

// Defines Trig and Echo pins of the Ultrasonic Sensor
HCSR04 sensor = HCSR04(D9, D8);
// Define pin for Servo
Servo myServo(D10);
// Define pins for LEDs
DigitalOut greenLED(D4);
DigitalOut redLED(D2);

Another important thing to do is to setting up the bluetooth part and the two needed services:

void bleInitComplete(BLE::InitializationCompleteCallbackContext *params) {
    BLE& ble = params->ble;
    ble_error_t error = params->error;
  
    if (error != BLE_ERROR_NONE) {
        onBleInitError(ble, error);
        return;
    }
  
    if (ble.getInstanceID() != BLE::DEFAULT_INSTANCE) {
        return;
    }
  
    ble.gap().onDisconnection(disconnectionCallback);
  
    // Setup distance and angle services
    uint8_t distance = 60;
    DistanceService distanceService(ble, distance);
  
    uint8_t angle = 15;
    AngleService angleService(ble, angle);
  
    // Setup advertising
    ble.gap().accumulateAdvertisingPayload(GapAdvertisingData::BREDR_NOT_SUPPORTED | GapAdvertisingData::LE_GENERAL_DISCOVERABLE);
    ble.gap().accumulateAdvertisingPayload(GapAdvertisingData::COMPLETE_LIST_16BIT_SERVICE_IDS, (uint8_t *)uuid16_list, sizeof(uuid16_list));
    ble.gap().accumulateAdvertisingPayload(GapAdvertisingData::COMPLETE_LOCAL_NAME, (uint8_t *)DEVICE_NAME, sizeof(DEVICE_NAME));
    ble.gap().setAdvertisingType(GapAdvertisingParams::ADV_CONNECTABLE_UNDIRECTED);
    ble.gap().setAdvertisingInterval(50);
    ble.gap().startAdvertising();

Finally we have to handle the sensors and the services, sending the data to the android app via bluetooth.NB: it's an infinite loop, but it works only if a smartphone is connected via bluetooth (if branch). If not, no sensor or service activity are performed, but it just wait for connection (else branch).

while (true) {
    // check for trigger from periodicCallback()
    if (triggerSensorPolling && ble.getGapState().connected) {
        triggerSensorPolling = false;
  
        // rotates the servo motor
        myServo.SetPosition(500 + (angle * 12.1212));
        distance = calculateDistance();
        if(distance < dangerDistance) {
            greenLED = LOW;
            redLED = HIGH;
        } else {
            redLED = LOW;
            greenLED = HIGH;
        }
        distanceService.updateDistanceValue(distance);
        angleService.updateAngleValue(angle);
        if(increment) angle++;
        else angle--;
        if(angle == 15) increment = true;
        else if(angle == 165) increment = false;
    } else {
        ble.waitForEvent(); // Low power wait for event
    }
}

On the Github page it's possible to find the enteire code.

3 - Application Development

For the Android application we started from scratch and we built a single activity app.The application was developed for Android smartphones with version 6.0+ (Sdk 23) as this version requires special permissions to use geolocation, necessary to scan BLE devices, and it was tested on a smartphone running Android 9.0. The user interface (UI) is very simple and user-friendly. You can see the radar in the main page (Home) and it's possible to configure and connect to nucleo through settings page.

It checks for Geolocalization permission at runtime, and asks to turn on the Bluetooth and the geolocalization if they are disabled.

It's possible to change some default variables in the code:

• nucleoMac: the MAC Address of Bluetooth nucleo board;
• presenceTreshold: threshold after which the alert is sent to the bot;
• rangeDistance: distance below which the potential danger counter is increased;
• countdownTimer: time needed to send the alert to the bot and in which it is possible to stop the sending of the alert.

private final String nucleoMAC = "XX:XX:XX:XX:XX:XX";
private int presenceTreshold = 20, 
private int rangeDistance = 40, 
private int countdownTimer = 10;

Processing has been used to draw the radar. The Sketch class extends the default processing PApplet class and here the radar animation is implemented.

In the GattAttributes class the used services and characteristics are defined:

public static String DISTANCE_MEASUREMENT = "00009a99-0000-1000-8000-00805f9b34fb";
public static String ANGLE_MEASUREMENT = "00009a90-0000-1000-8000-00805f9b34fb";
public static String DISTANCE_SERVICE = "0000990d-0000-1000-8000-00805f9b34fb";
public static String ANGLE_SERVICE = "0000990f-0000-1000-8000-00805f9b34fb";

The main feature is to connect to the Nucleo board. So when you press the Connect button in the settings tab, the smartphone starts to scan all BLE devices around. If it finds the Nucleo board (thanks to its MAC address), it stops scanning devices and connects to the board.

// Function that starts the scanning of Nucleo device
public void startScanning() {
    textview.setText("");
    AsyncTask.execute(new Runnable() {
        @Override
        public void run() {
            if(btScanner == null) btScanner = btAdapter.getBluetoothLeScanner();
            btScanner.startScan(leScanCallback);
        }
    });
}
  
// Device scan callback.
private ScanCallback leScanCallback = new ScanCallback() {
    @Override
    public void onScanResult(int callbackType, ScanResult result) {
        BluetoothDevice device = result.getDevice();
        if(device.getAddress().equals(nucleoMAC)) { // Nucleo device found
            btScanner.stopScan(leScanCallback);
            textview.append("Connected to: " + device.getName()+"\n");
            btGatt = device.connectGatt(getApplicationContext(), false, bleGattCallback);
        }
    }
};

After the connection it's necessary to read data from Nucleo board. To use our services and characteristics, setting the radar and check if there is a threat behind:

// Function that receives data from Nucleo board
private void receiveData(final BluetoothGattCharacteristic characteristic) {
    // Receiving distance
    if (UUID_DISTANCE_MEASUREMENT.equals(characteristic.getUuid())) {
        int format = BluetoothGattCharacteristic.FORMAT_UINT8;
        final int distance = characteristic.getIntValue(format, 0);
        sketch.setDistance(distance);
        if(distance < rangeDistance) numOfPresence++;
        checkThreat();
    }
    // Receiving angle
    else if (UUID_ANGLE_MEASUREMENT.equals(characteristic.getUuid())) {
        int format = BluetoothGattCharacteristic.FORMAT_UINT8;
        final int angle = characteristic.getIntValue(format, 0);
        sketch.setAngle(angle);
        if(angle == 0 || angle == 165) {
            numOfPresence = 0;    //reset counter of detection
            sentAlert = false;
        }
    }
}

If a potential danger has been detected behind you, it's necessary to send the location and name of the user in danger to the telegram bot:

private void sendToTelegram(){
    String url;
    String name = preferenceManager.getString("name", null);
    if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.ACCESS_FINE_LOCATION) == PackageManager.PERMISSION_GRANTED            || ContextCompat.checkSelfPermission(this, android.Manifest.permission.ACCESS_COARSE_LOCATION) == PackageManager.PERMISSION_GRANTED) {
        double latitude = 0, longitude = 0;
        LocationManager locManager = (LocationManager)getSystemService(Context.LOCATION_SERVICE);
        locManager.requestLocationUpdates(LocationManager.GPS_PROVIDER, 1000L, 500.0f, MyLocation.locationListener, Looper.getMainLooper());
        Location myLocation = MyLocation.getLastKnownLocation(getApplicationContext());
        if (myLocation != null) {
            latitude = myLocation.getLatitude();
            longitude = myLocation.getLongitude();
            url = "https://guarded-mountain-88932.herokuapp.com/notification?device_id=deviceId&name=" + name +"&lat=" + latitude + "&lon=" + longitude;
        }
        else url = "https://guarded-mountain-88932.herokuapp.com/notification?device_id=deviceId&name=" + name;
    }
    else url = "https://guarded-mountain-88932.herokuapp.com/notification?device_id=deviceId&name=" + name;
    RequestQueue ExampleRequestQueue = Volley.newRequestQueue(activity);
    StringRequest ExampleStringRequest = new StringRequest(Request.Method.GET, url, new Response.Listener<String>() {
        @Override
        public void onResponse(String response) {
            //This code is executed if the server responds, whether or not the response contains data.
            //The String 'response' contains the server's response.
        }
    }, new Response.ErrorListener() { //Create an error listener to handle errors appropriately.
        @Override
        public void onErrorResponse(VolleyError error) {
            //This code is executed if there is an error.
        }
    });
    ExampleRequestQueue.add(ExampleStringRequest);
    timerView.setText("Alert sent!");}

Here it's possible to see the application UI:

4 - Telegram Bot

4 - 1 - Register the bot:

Register a Telegram bot is done via another bot on telegram!

So you need to add on Telegram @BotFather, it is the designated robot for registering and managing your bots. Simply start chatting with it and you will see all of your options. Here we focus only on registering our new bot.

Simply type:

/newbot

and you will be asked to provide a name and identifier for your bot. Once done, @BotFather will give you a TOKEN which is what you need to configure in your server component later. This token shouldn't be public, so keep it in safe. But if the token has been compromised, then speak with @BotFather again to change it.

4 - 2 - Bot server:

At this point install the required modules, here we use standard http module for handling HTTP request from the device and also we need some how to interact with Telegram Bot API. Well, we could send and receive events and messages from telegram API by sending raw HTTP request using aforementioned module. But, It's far more simpler to use node-telegram-bot-api node module which provides high level client API.

npm init
npm install --save node-telegram-bot-api

4 - 3 - Build HTTP endpoint and Bot server

This is how the node.js HTTP endpoint has been implemented:

const http = require('http');
const TelegramBot = require('node-telegram-bot-api');
const {
   createEchoHandler,
   createNotificationServerHandler,
   createTierHandler,
} = require('./handlers');
const TOKEN = process.env.TELEGRAM_TOKEN;
const options = {
   polling: true
 };
const bot = new TelegramBot(TOKEN, options);
bot.onText(/\/echo (.+)/, createEchoHandler(bot));
bot.onText(/\/tie (.+)/, createTierHandler(bot));
http.createServer(createNotificationServerHandler(bot)).listen(process.env.PORT);  
  

Where in the module called handlers.js you will have the functions you need. So the function createEchoHandler returns an Bot's handler to debug the server with the common echo message technique:

function createEchoHandler(bot) {
   return function onEcho(msg, match) {
       bot.sendMessage(msg.chat.id, match[1]);
   }
}

The function createTierHandler returns another Bot's callback handler to link a user to a particular device with a username:

function createTierHandler(bot) {
   return function onTie(msg, match) {
       const deviceId = match[1];
       db.setChatId(deviceId, msg.chat.id, (r) => {
           if (r && r.value) {
               bot.sendMessage(msg.chat.id, `Done!`);
           } else {
               bot.sendMessage(msg.chat.id, `Device with ${deviceId} is not found 😢`);
           }
       });
   }
} 

The function createNotificationServerHandler returns an HTTP callback for all incoming requests. That callback listen for broadcasting events to send an alarm message to all users:

function createNotificationServerHandler(bot) {
   return function onDeviceNotification(req, res) {
       const { pathname, query: q } = url.parse(req.url, true);
       if (req.method === 'GET' && pathname === '/notification' && q && q.device_id) {
           db.getDocByDeviceId(q.device_id, (doc) => {
               if (doc) { // send message to moltiple receivers with chat id
                   doc.chat_ids && doc.chat_ids.forEach(id => {
                       const name = q.name;
                       bot.sendMessage(id, createInfoMsg(doc, name));
                       if (q.lat && q.lon) bot.sendLocation(id, q.lat, q.lon);
                   });
                   res.writeHead(200, { "Content-Type": "text/plain" });
                   res.end(doc.chat_ids ? 'Ok, notified!' : 'Client is not connected!');
               } else {
                   res.writeHead(400, { "Content-Type": "text/plain" });
                   res.end(`Cannot find device with id:${q.device_id}! 😢`);
               }
           });
       } else {
           res.end(`Unknown request!`);
       }
   }
}
/**
* Creates notification message 
* @param {Object} doc - document from MongoDB collection
* @param {String} name - name of a user
* @returns {String}
*/
function createInfoMsg(doc, name) {
   return `❗ - device: ${doc.device_id};${name ? '\nname: ' + name : ''}`;
} 

And in the module repository.js you will find all the functions related to MongoDB server interaction: setChatId to attach telegram users chat with 'chatId' to the particular device with 'deviceId', and getDocByDeviceId to retrieve a document for a particular device by its id.

  
/**
* Attach telegram user's chat with 'chatId' to the particular device with 'deviceId'
* @param {String} deviceId
* @param {String} chatId
* @param {Function} cb - callback
*/
function setChatId(deviceId, chatId, cb) {
   if (!col) {
       cb(null);
       return;
   }
   getDocByDeviceId(deviceId, (doc) => {
       if (!doc) {
           cb(null);
           return;
       }
       console.log('DOC: ', doc);
       chatIds = doc.chat_ids.concat(chatId);
       col.findOneAndUpdate({device_id: deviceId}, {$set: {chat_ids: chatIds}}, (err, r) => {
       if (err) {
           console.log(err);
           cb(null);
       } else {
           cb(r);
       }
   })});    
}
  
  
function getDocByDeviceId(deviceId, cb) {
   col && col.find({ device_id: deviceId }).limit(1).next((err, doc) => {
       if (err) {
           console.error(err); 
           cb(null);
       } else {
           cb(doc);
       }
   });
} 

Before this you need to call a connect function to connect client to a MongoDB instance:

client.connect((err) => {
   if (err) throw err;
   console.log("Connected successfully to server");
   db = client.db(dbName);
   col = db.collection(colName);
});

Where dbName and colName are your database and collection name accordingly.

4 - 4 - Deploy on Heroku

heroku create

This will create a new app in your heroku account (you will be asked to setup one if have no account setup) with a random name. To open the app in your browser simply run:

heroku open

Each time you need to deploy the app, simply run the following command and push your latest code to heroku remote repo, which then results in your node.js app to build and start on Heroku server.

git push heroku master

For the sake no commercial use, a free Heroku account has been used with 496 MB as max space that is more than enough for our purpose.

Video demo

Linkedin

• Marco Costa
• Giuseppe Capaldi
• Artem Savchuck