Created August 6, 2020 © CC BY-NC-SA

City Data Tracker

Collect air quality, weather, road, and other data using a low power device mounted to your vehicle

AdvancedFull instructions provided20 hours894

Things used in this project

Hardware components

Helium Developer Kit

Software apps and online services

Amazon Web Services AWS IoT

Helium Console

Amazon Web Services AWS Lambda

MongoDB

Story

Data has the potential to change the way we look at issues and solve problems. Collecting anonymous data about the environment around us can benefit our society greatly.

Weather data is often collected by a few weather stations situated sparsely around city environments, resulting in inaccurate data for many people who depend on the data to plan their day. Roads are often bumpy, filled with potholes and cracks, resulting in potential vehicle damage and increasing the risk of accidents. Many of these issues are related to a lack of adequate data collection.

The solution to this: IoT devices attached to vehicles connected to the Helium Network.

Using the Helium Developer Kit and its available sensors, I created a proof of concept to use the Helium Network to collect relevant data and display it to potential users.

I first implemented the ability to detect potholes and record the corresponding GPS location. Using the Helium Developer Kit's IMU and GPS modules, I wrote a script to monitor if there is excessive and sudden Z (vertical) axis movement. When this movement is detected (corresponding to a significant bump in the road), the GPS module records the current GPS location. Then, all of this data is sent via the Helium Console to a database using AWS IoT and AWS Lambda so that it can be viewed by trusted parties in the future.

I have included my script in the code below. To set this up, I created a new lambda script and uploaded the dependencies as a zip file. I then used MongoDB Atlas to set up the backend so that the data can be sent there, making sure to paste the username and password into my Lambda environment variables. The script takes data sent by the Helium Console through AWS IoT and processes it prior to sending it to the database. To set up the connection from the console to the lambda script, I followed this guide: https://developer.helium.com/console/integrations/awsiotcore

I also used the onboard temperature, pressure, and humidity sensors to collect weather related data every few minutes, also recording the GPS location where these measurements were taken so that many data points can be compiled together to make a better, more accurate weather map. Again, this data is sent via the Helium Console to a database using AWS IoT and AWS Lambda.

Data in MongoDB

The Helium Network allows data to be collected and sent in almost real-time from many different vehicles. In the future, this system can be expanded to use many sensors on many different vehicles to build comprehensive datasets around both cities and rural areas that can be used by companies and voernments to better inform their decisions.

AWS Lambda Script

import json
import base64
import os
import math
import pymongo
from datetime import datetime

# from botocore.vendored import requests
mongoURL = "mongodb+srv://" + os.environ['MONGO_USERNAME'] + ":" + os.environ['MONGO_PWD'] + "@" + os.environ['MONGO_URL'] + "?retryWrites=true&w=majority"
client = pymongo.MongoClient(mongoURL)
db = client["test"]
collection = db["sensor"]

def lambda_handler(event, context):
    print(event['payload'])
    decoded = base64.b64decode(event['payload'])
    print(decoded)
    print(list(decoded))
    data = list(decoded)
    count = 0
    outDict = {}
    while count+2 < len(data):
        dataID = str(data[count])
        dataType = data[count+1]
        if dataType == 3: # Analog output
            dataNum = data[count+2] + data[count+3] * 0.01
            print(dataNum)
            count = count + 4
            outDict[dataID] = {"type": "ANALOG", "data": dataNum}
        elif dataType == 136: # GPS Loc
            parta, neg = process_hex(str(hex(data[count+2]))[-2:], partA=True)
            partb, throwaway = process_hex(str(hex(data[count+3]))[-2:])
            partc, throwaway = process_hex(str(hex(data[count+4]))[-2:])
            total = parta + partb + partc
            print(total)
            lat = int(total, 16)
            if neg:
                print(neg)
                lat = lat - int(neg, 16)*math.pow(16, 5)
                # DO STUFF
            lat = lat/10000
            print(lat)

            parta, neg = process_hex(str(hex(data[count+5]))[-2:], partA=True)
            partb, throwaway = process_hex(str(hex(data[count+6]))[-2:])
            partc, throwaway = process_hex(str(hex(data[count+7]))[-2:])
            total = parta + partb + partc
            lon = int(total, 16)
            if neg:
                print(neg)
                lon = lon - int(neg, 16)*math.pow(16, 5) + (int(neg, 16) - 16)*math.pow(16, 5)
            lon = lon/10000
            print(lon)

            parta, throwaway = process_hex(str(hex(data[count+8]))[-2:])
            partb, throwaway = process_hex(str(hex(data[count+9]))[-2:])
            partc, throwaway = process_hex(str(hex(data[count+10]))[-2:])
            total = parta + partb + partc
            # print(total)
            alt = int(total, 16)/100
            outDict[dataID] = {"type": "GPS", "data": {"lat": lat, "lon": lon, "alt": alt}}
            count = count + 11
        elif event['payload'] == 'VGltZXIA': # Heartbeat
            print("Heartbeat")
            break
    print(outDict)
    if outDict:
        outDict['time'] = datetime.utcnow()
        collection.insert_one({"data": outDict, "time": datetime.utcnow(), "deviceID": "1"}) # TODO FIX DEVICE ID
    return 0

def process_hex(hexVal, partA=False):
    print(hexVal)
    if hexVal[0] == 'x':
        print("A", hexVal)
        return '0' + hexVal[1], 0
    elif partA and int(hexVal[0], 16) > 7:
        print("B", hexVal)
        return hexVal, hexVal[0]
    else:
        print("C", hexVal)
        return hexVal, 0

Credits

Jerry Zhang

11 projects • 15 followers

City Data Tracker

Things used in this project

Hardware components

Software apps and online services

Story

Code

AWS Lambda Script

Credits

Jerry Zhang

Comments

Embed the widget on your own site

City Data Tracker

City Data Tracker

Things used in this project

Hardware components

Software apps and online services

Story

Code

AWS Lambda Script

Credits

Jerry Zhang

Comments

Related channels and tags