COVPAY: SMS Scheduling and Surveillance For Businesses

Story

One of the biggest issue with COVID-19 is logistics and shopping. In many developing countries high demand has increased lines and transmission rates. A recent case study is in India where the demand for alcohol led to high lines with no planing and increased transmission.

Many developing countries have access to an SMS device. The solution to this problem is an SMS based logistics planner to automatically schedule timings for users to buy and/or pick up goods at their store. Store owners (communal) share the device and users simply text the shop they require and the time intervals that best suites them via a special SMS message. The device automatically provides them with a booking (if available or otherwise schedules the next most appropriate time). Along with the SMS technology, the device also provides real-time motion detection to act as a security/surveillance system with SMS alerts, as well as streaming environmental data via the ThunderBoard Sense 2 to determine environmental conditions (indoor or outside) The design provides a low-cost community solution for the intended device, coping with the high demand solutions in a safer and automated way. The design helps to prevent the spread of COVID-19

Step 1: AWS Chatbot

Create your AWS Chatbot and then create an Amazon Lex service and choose the custom option.

Create the first intent and create a new slot with the name inventory. Add all the items you have.

Then add the entry responses (utterances) and add all suitable questions (re-worded).

Adding slots are very important, the order is set out as below:

For the date section click on the settings bar and add the following data

For the PickupTime add the following data

Finally add the confirmation prompts

Build and Publish the bot and test out to see if it is fully functional

Step 2: Connect To Twilio

Build and publish the bot to twilio by adding the correct user data

After activating you will be able to generate and endpoint. Copy this endpoint and navigate to Twilio. There find your phone numbers and under the messages section add the endpoint url

Step 3: Testing The Chatbot

The SMS can be viewed directly either in the transcripts on Twilio or on the BalenaFin if you have the Sim Card Inside.

Step 4: Camera for BalenaFin

The solution needs to be cost effective and to ensure this, it must have multiple features. One issue is adding security to shops as many are in a position where setting up a camera system is difficult. For this part of the project you need to have rasbian installed onto the BalenaFin

I) Add Libraries

sudo apt-get update

Open CV2

sudo apt-get install python-opencv

Numpy

sudo apt-get install python3-numpy

Requests

sudo apt-get install python-requests

Add Camera using:

sudo raspi-config

Then navigate and enable the camera. Then to test the camera run:

raspistill -v -o test.jpg

If it does not work double check the raspberry pi camera to the BalenaFin and try again.

Step 5: Azure Trigger

For ease of development we will use Azure to call and send the SMS when motion is detected, if you want to use AWS follow this: https://www.twilio.com/docs/sms/tutorials/how-to-receive-and-reply-python-amazon-lambda

I) Create a new logic app and add a https trigger (when a http request is received)

II) Navigate to the resource and add a GET method

III) Add a new step, search and select Twilio Send Text Message and fill in all details

IV) Hit save and copy the http code. Run the logic app to determine it fully works

Step 6: Programming Motion Detector

Download and run this python program on the Balena Fin

import cv2
import requests
import numpy as np
cap= cv2.VideoCapture(0) # 0 means camera 
ret, frame1 = cap.read()
ret, frame2 = cap.read()

while cap.isOpened():
    diff= cv2.absdiff(frame1,frame2)
    gray=cv2.cvtColor(diff, cv2.COLOR_BGR2GRAY)
    blur= cv2.GaussianBlur(gray,(5,5),0)
    _, thresh= cv2.threshold(blur, 20,255,cv2.THRESH_BINARY)
    dilated=cv2.dilate(thresh, None, iterations=3)
    contours, _ = cv2.findContours(dilated, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    for contour in contours:
        (x, y, w, h) = cv2.boundingRect(contour)

        if cv2.contourArea(contour)< 700:
            continue
        cv2.rectangle(frame1, (x,y), (x+w, y+h), (0,255,0), 2)
        cv2.putText(frame1, "Status:{}".format('Movement'), (10,20), cv2.FONT_HERSHEY_COMPLEX,
                                                   1,(0,0,255),3)
	requests.get("ENTER HTTP HERE")

    #cv2.drawContours(frame1,contours,-1,(0,255,0),2)
    cv2.imshow ("feed",frame1)
    frame1=frame2
    ret, frame2=cap.read()
    if cv2.waitKey(40) == 27:
        break

cv2.destroyAllWindows()
cap.release()

Step 7: [OPTIONAL] Collecting Environmental Data

Depending on the placement of the device environmental data might want to be collected. For this part we will interface using ThingSpeak and the Thunderboard Sense 2.

I) When you first open the thunderboard sense 2, download the android or ios app and connect to your device. This will automatically stream data from the device to the phone. This will help verify all aspects of your device is working. Also 3D Print the casing attached in the files.i

II) Update libraries on your Balena Fin by typing

sudo apt update
sudo apt upgrade

III) Install Python 3

sudo apt-get install python3-pip

IV) For BLE connectivity install bluepy (licensed under GNU Public V2) using:

sudo apt-get install python-pip libglib2.0-dev
sudo pip install bluepy

If you are not running a version of Debian follow the instructions here: https://github.com/IanHarvey/bluepy

While everything is downloading feel free to move onto the next step

Setting up ThingSpeak

I) Log on/Create an account and then set up a new channel

II) Add 5 Fields in this order: Battery, Temperature, Humidity, UV Index, Carbon Dioxide

II) Find your Write API Key by navigating to API Keys section. It should look something like this with each x being a number or letter: "xxxxxxxxxxxxxxxx"

Setting up Thunderboard Sense 2

I) Download the Bluetooth Sniffer App

II) Power your thunderboard and find it in the sniffer app and click connect

III) You should be able to see your device's ID/Address please note this as you will need it in the program

Programming Balena Fin

Download the program attached/below and change:

i) The API Key

ii) Device Address

Then add it to the Balena Fin and open up the terminal and run the program

from __future__ import division
import sys
from bluepy.btle import *
import struct
import thread
from time import sleep
import urllib2
PRIVATE_KEY = 'REPLACE WITH KEY'
groundURL = 'https://api.thingspeak.com/update?api_key='
def StreamingSensor():
scanner = Scanner(0)
devices = scanner.scan(3)
for dev in devices:
print "Device %s (%s), RSSI=%d dB" % (dev.addr, dev.addrType, dev.rssi)
for (adtype, desc, value) in dev.getScanData():
print "  %s = %s" % (desc, value)
num_ble = len(devices)
print num_ble
if num_ble==0:
return None
ble_service = []
char_sensor = 0
non_sensor = 0
bat_char = Characteristic
temperature_char = Characteristic
humidity_char = Characteristic
uvindex_char = Characteristic
co2_char = Characteristic
count = 15
for i in range(num_ble):
try:
devices[i].getScanData()
ble_service.append(Peripheral())
ble_service[char_sensor].connect('REPLACE WITH BLUETOOTH ADDRESS',devices[i].addrType)
#ble_service[char_sensor].connect(devices[i].addr, devices[i].addrType)
char_sensor = char_sensor + 1
print "Connected %s device with addr %s " % (char_sensor, devices[i].addr)
except:
non_sensor = non_sensor + 1
try:
for i in range(char_sensor):
services = ble_service[i].getServices()
characteristics = ble_service[i].getCharacteristics()
for k in characteristics:
print k
if k.uuid=="2a19":
print "Battery Level"
bat_char = k
if k.uuid == "2a6e":
print "Temperature"
temperature_char = k
if k.uuid == "2a6f":
print "Humidity"
humidity_char = k
if k.uuid == "2a76":
print "uvindex"
uvindex_char = k
if k.uuid == 'efd658ae-c401-ef33-76e7-91b00019103b':
co2_char = k
except:
return None
while True:
bat_data = bat_char.read()
bat_data_value = ord(bat_data[0])
temperature_data = temperature_char.read()
temperature_data_value =(ord(temperature_data[1])<<8)+ord(temperature_data[0])
float_temperature_data_value = (temperature_data_value / 100)
humidity_data = humidity_char.read()
humidity_data_value =(ord(humidity_data[1])<<8)+ord(humidity_data[0])
uvindex_data = uvindex_char.read()
uvindex_data_value = ord(uvindex_data[0])
co2_data = co2_char.read()
co2_data_value = ord(co2_data[0])
print "Battery: ", bat_data_value
print "Temperature: ", float_temperature_data_value
print "Humidity: ", humidity_data_value
print "UVIndex: ", uvindex_data_value
print "CO2: ", co2_data_value
if count > 14:
f = urllib2.urlopen(groundURL + PRIVATE_KEY +"&field1=%s&field2=%s&field3=%s&field4=%s&field5=%s" % (bat_data_value, float_temperature_data_value, humidity_data_value,uvindex_data_value,co2_data_value))
print f.read()
f.close()
count = 0
count = count + 1
sleep(1)
while True:
StreamingSensor()

Code adapted from https://www.hackster.io/neal_markham

sudo python REPLACE WITH FILE ADDRESS

Please note you MUST be in sudo for this to work properly. If it does not connect unplug the thunderboard sense 2, re-plug it wait 3 seconds and promptly run the program

Data Collection

If you look onto thingspeak you will the data is being updated live

Conclusion

Overall, this design helps to prevent the spread of COVID-19 through the universal SMS based shopping scheduling bot which helps to protect against the spread and limit issues of over-crowding, long lines etc. On top of this the design provides surveillance and motion alerts for shops to protect against robberies and unwelcome guests, whilst simultaneously providing users with environmental data to help determine the conditions the store is surrounded with. This device helps businesses and shop-owners with a highly accessible system which can help with logistics.