DRONE with AT&T and PubNub IoT Connectivity

Drones are being used to replace much more expensive commercial and industrial inspection methods requiring using helicopters or the construction of scaffolding.

To use a drone in commercial and industrial settings often requires extensive design work to make the drone extremely reliable and able to function in adverse conditions. In my prior Hackster.io drone project, I demonstrated a system for detecting propeller blade hits on inspection drones.

https://www.hackster.io/agent-titanium-c6063b/flexio-based-multi-copter-rotor-control-57d124?ref=challenge&ref_id=32&offset=0

Keeping inspection customers happy and keeping drones healthy and intact are topics very much on the mind of this inventor.

For commercial inspection work there is a desire to convey inspection results quickly, so an LTE modem would be ideal for this task.

There are also opportunities to monitor the vital signs of the drone to make sure that the flights are safe flights, that generate positive results and only favorable publicity.

AT&T IoT Starter Kit and PubNub to the rescue!

The AT&T IoT Starter kit includes an LTE modem which I used to extend my drone’s functionality beyond that of a typical drone, to provide a backup for the radio control link, and to safely guide the drone to a landing zone in the event of a failure of the radio control link.

To Do List before the AT&T Starter Kit Arrives

• Create AT&T Flow Designer Account

• Create AT&T M2X Developer Account

• Create PubNub Account

• Mess around with tutorials a bit.

Putting something together very quickly for an early Dec 22 project submission

Building on the Starter Kit Example Projects

There was just not enough time to do anything terribly fancy, so I am modified the starter kit example to meet my project goals. This also fitted nicely with the iterative design process enabled by the AT&T Flow programming environment.

This tutorial demonstrates the collection of sensor data with M2X and the Flow programming environment. It demonstrates the trigger feature which I want to use in my project. Fortunately the MBED project that supplies the firmware for the NXP Freedom K64 board has the ability to send Accelerometer and GPS data streams. I also sent along temperature to for a nice beginning to creating a drone telemetry stream.

https://starterkit.att.com/tutorials/starter-kit-guide

https://flow.att.io/starter-kit-core/starter-kit-base/home

But we also have resources supplied though PubNub that I wanted to learn how to leverage.

https://www.pubnub.com/blog/2016-11-16-pubnub-and-the-att-iot-starter-kit-an-introduction-to-integrating-pubnub-on-the-att-client/

Mixing these two software example projects together created the AT&T PubNub IoT enabled Drone software project!

Xadow GPS module

The website for the starter kit, www.starterkit.att.com, has a tutorial for extending the base starter kit project to use a Xadow GPS module. This is exactly what I needed to get done next so I went through the tutorial.

https://starterkit.att.com/tutorials/extending-the-starter-kit-project-with-the-xadow-gps-module

Things to know about the PubNub MBED API

https://www.pubnub.com/docs/mbed/data-streams-publish-and-subscribe

Speeding up the programming effort

Efforts were focused on sending "dummy" telemetry reports from the AT&T IoT Starter Kit into the cloud, and working on the Flow programming to get cool things to happen in response to the "dummy" telemetry. It did not make much sense to make the drone physically real (and drone telemetry real) until the cloud programming was both correct and interesting.

The Flow Program for the Drone

The program above is, for the moment, copying the telemetry back to the drone. The test string is "temp: 50". So the "small matter of programming" or SMOP is to change the check_telemetry function from the present pass the data through unchanged process to something much more interesting.

This shows that telemetry data has indeed been sent back to the AT&T IoT Starter Kit. The data has gone from the drone up to the cloud and come right back to the drone.

Here is the JavaScript for the check_telemetry function. The message input to the function is the output of the function.

The really scary part of the project is behind us. It is a matter of building the check_telemetry function to do real command and control sorts of things. There is the ability to inject string data so we can build and test without having to use the LTE modem to convey the data.

Getting the real live sensors working will be no problem we have many code examples to show how to do that.

Modifying the MBED program, that runs on the AT&T IoT Starter Kit, to interpret the command and control strings received by the LTE modem will be no problem either.

Injecting Data in Flow

Adding an inject node allows us to test the Flow program without having to use the IoT Starter Kit to transmit the data. This will be extremely useful for testing various responses to the incoming IoT drone telemetry.

Simulating an ERROR

Using another inject node I will inject a simulated temperature spike. This will be caught in my AT&T Flow design as a trigger event that causes a series of GPS coordinates to be sent from the cloud to the IoT based drone, to indicate a safe landing approach. The landing zone is in the center of a traffic circle near my workplace. I really would not use this as a landing zone in “real life”.

A very simple flow program to detect a reported temperature over 79 degrees Celsius. If the temperature is under 79 degrees Celsius "ALL SYS GO!" is passed back to the IoT drone. If the temperature is above 79 degrees Celsius a safe landing approach is passed back to the IoT drone. The debug output shows the GPS coordinates of a ten GPS waypoint landing approach.

Here is the JavaScript for the improved check_telemetry function.

var tempValue = parseFloat(msg.payload[8]);//msg.payload[0]
var safe1 = "SAFETY! 39.961433 -86.127472 290,";
var safe2 = " 39.961436 -86.127388 280,";
var safe3 = " 39.961440 -86.127304 270,";
var safe4 = " 39.961442 -86.127220 260,";
var safe5 = " 39.961444 -86.127136 250,";
var safe6 = " 39.961446 -86.127052 240,";
var safe7 = " 39.961448 -86.127014 230,";
var safe8 = " 39.961450 -86.126970 220,";
var safe9 = " 39.961450 -86.126970 198,";
var safe10 = " 39.961450 -86.126970 197,";
if (tempValue >= 1) { //assume something was found
       msg = {}; //Create blank object
       msg.payload = (tempValue >= 8) ? safe1+safe2+safe3+safe4+safe5+safe6+safe7+safe8+safe9+safe10 : "ALL SYS GO!";     
}
return msg;

http://www.movable-type.co.uk/scripts/latlong.html

This is a very good link for background on the math involved in GPS related calculations.

GPS enforced flight boundaries

In a similar fashion to the temperature based trigger event, a second development milestone will be to provide trigger events based on monitoring the GPS location data stream.

The first test will be to provide a cube shaped safe flight zone.

There are high and low limits placed on altitude, longitude and latitude data from the GPS. These limits taken together form a safe flight zone cube. Leaving the safe flight zone will also cause a series of GPS coordinates to be sent from the cloud to the IoT based drone, to indicate a safe landing approach. A similar test to the temperature spike test will be used to test GPS enforced flight boundaries.

Here is the Flow diagram for the program. There are a few more nodes for injecting strings for testing the system. The Flow code is checking GPS enforced flight boundaries and temperature.

Here is the further improved check_telemetry function that enforces GPS and altitude boundaries. These are my first programming attempts in JavaScript and it probably shows. I spend most of my time coding in C, writing code for embedded systems

var tempValue = parseFloat(msg.payload.substring(8,11));
var lat_delta = 0.005;
var lon_delta = 0.005;
var ceiling = 100; // 100 meters over the ground
var ground = 258;
var ref_lat = 39.961282;
var ref_lon = -86.127067
var safe1 = "SAFETY! 39.961433 -86.127472 290,";
var safe2 = " 39.961436 -86.127388 280,";
var safe3 = " 39.961440 -86.127304 270,";
var safe4 = " 39.961442 -86.127220 260,";
var safe5 = " 39.961444 -86.127136 250,";
var safe6 = " 39.961446 -86.127052 240,";
var safe7 = " 39.961448 -86.127014 230,";
var safe8 = " 39.961450 -86.126970 220,";
var safe9 = " 39.961450 -86.126970 198,";
var safe10 = " 39.961450 -86.126970 197,";
msg2 = {};
// assume that there will be a fault found
msg2.payload = safe1+safe2+safe3+safe4+safe5+safe6+safe7+safe8+safe9+safe10; 
if (tempValue >= 1) { //assume something was found
       if (tempValue >= 80){
       }else{
           tempValue = parseFloat(msg.payload.substring(15,26));
           if(  (ref_lat - lat_delta) < tempValue ){
               if(tempValue < (ref_lat + lat_delta)){
                   tempValue = parseFloat(msg.payload.substring(27,38));
                   if(  (ref_lon - lon_delta) < tempValue ){
                       if(tempValue < (ref_lon + lon_delta)){
                           tempValue = parseFloat(msg.payload.substring(39,47));
                           if(ground >= tempValue){
                               if( tempValue < (ground + ceiling)){    
                                   msg2.payload = "ALL SYS GO!";
                               }
                           }
                       }
                   }
               }
           }
       }
}
return msg2;

In a system developed further the IoT drone could be protected from other objects moving in the IoT drone's workspace, the flight boundaries could be constantly adjusted by decisions made by cloud based programs.

Further work

I hope to have further projects where dashboards for drone telemetry and maps showing flight paths appear, perhaps with images of what the drone captures with a camera. There are so many possibilities with the cloud based services that can be leveraged with PubNub!

When people say drone more often they mean something that flies. More generally it can include any moving object, so what I am learning can be applied to robots as well.