Cloud Controlled Robot Brain

Introduction:

The combination of Hologram Nova USB Modem and Raspberry Pi Zero W will bring about a game changing innovation that could revolutionised how innovators, gadget designers and hardware hackers use "Nearby Communication" to provide a World-wide device-to-device (M2M) data visualisation, monitoring for connected Robots, Drones, Submarine, Agricultural devices and products. The technology could also be use for indoor location based services in large retail or shopping mall, for example it could be for monitoring of items on each store floor level (e,g. 1st floor, 2nd floor, etc...) or to direct customers to item location when selected on their mobile phones or virtual screen. Other possible areas are: Home monitoring, Elderly assistant and care, Asset tracking, in-Car infotainment, Amusement Park Guide and Ride, etc...

What we will be doing with Hologram Nova USB Modem and Raspberry Pi Zero W kits:

Deeplinking and Mapping API?

Deeplinking API enables use of HTTP/URL routines that allows locations and routes to be shared between users and devices being mapped. It makes it possible to track, follows and monitor connected devices in real-time and non real-time situations. It allow a Restful resources to share and be shared among connected located devices through a middle-ware relay server(s) to various subscribers (devices: Robots, Drones, etc...).

In this project, Hologram Nova USB Modem driven by Raspberry Pi Zero W Kit running ROS (Robotics Operating Systems) and a Cloud Based Machine Learning Software to act as the brain be deployed as "Location Based Services solution" for a Mobile Positioning System (MPS) which supports complementary positioning methods for 2G, 3G and 4G/LTE networks. This system will be mounted on a robot and a drone built for this competition.

Quick Glance:

A Quick mock-up of the project for Quad Copter wired with a Nova Modem in Raspberry Pi with a Motor Shield is shown below:

Hologram Nova USB Modem acting as cellular providing the location service for the Raspberry Pi Zero to and from the cloud via the gateway server in order to drive, monitor, control and exchange data from Robot and Drone mounted sensors (planned sensors are: range sensors, camera/vision tracker, pressure/altitude, temperature and infra-red)

Also shown below is that of Vex Robot

Connected devices can subscribe and also publish there presence, transmit data, link with other peers to collaborate on completion of task in real-time or on per-scheduled period irrespective of where they are located. Please see below for a typical scenario for a multi-robots all having their own Hologram-Nova cellular modem, which can communicate with Hologram Cloud, ground station and mobile app simultaneously.

Wide application areas: automation (any); collaborative factory robots, home automation, rescue missions, agriculture and farming, plantation, submarine, small-device-autonomous(e.g. RC racing cars), mining and tunnelling works, remote-repair, education and teaching and many more...

Three of the robots will be fitted with cameras for computer-vision and machine learning to showcase. One of the camera will be Flir based camera. The robot arm with be enhanced with ESP8266 having a webpage and remote access. The Vex robots will also have voice control feature.

I would have loved to install a Lidar to experiment but still a bit expensive.

A GitHub and YouTube will be made for the project.

Part-1: Getting Started with Hologram Nova kit:

The kit has arrived so I am unboxing here, going through the hardware assembly setupping the modem, Pi-Zero, etc... Our first unboxing short video is as shown below:

Shortly I will be doing the initial test drive to see if we can connect to the cloud server, send data while showing how you can explore the cloud-server and facilities that you can use. Cool and stay tuned in...

In order to showcase and demonstrates those advantages of using Hologram-Nova some areas that could give an edge for business, researchers, governments and the societies at large. the following were were conducted by me during the period of this competition.

Connecting to Cloud:

After setting up the hardware, get the operating into Pi-Zero, your next step is to go to at: https://hologram.io/ and create an account for your project, after-all the primary aim here is to send, analyse, collate and consume data in the cloud for business and profit, more about this later.... So you need an account

Next log into your account dashboard and setup your sim card details:

Next get the SDK install on your Pi-Zero, since it will be serving as Cloud Gateway for local devices that I have e.g. Robots, Lights, Sensors that were driven by ESP8266, STM SensorTile (running MicroPython with Mqtt client).

You need internet enabled and connected. Now issue the following commands to get the Hologram Python SDK installed:

curl -L hologram.io/python-install | bash

the above command will download and install the SDK on your Pi-Zero then you see the below message saying you are now ready to start using your SDK:

Make sure you have your nova modem kit connected to your Pi-Zero, WiFi is setup and internet access is ready:

Next issue the following command to see all possible things you can do with the SDK : We will be using the Pi-Zero Terminal from now on:

hologram -help 

You will be greeted with options shown below:

NOTE: The SDK can be extended with Python Language or through API that can interface with Python and more....

Let us send message to the cloud to see if we can connect to it and put data into the cloud :) Enter the command below:

sudo hologram send "Hello from Sanyaade"

Note: the word sudo above is required as the SDK is installed in restricted directories and the use of the nova modem is accessing hardware and dial-out in Unix like environment. So you need to be root okay!

If everything went well then you see this message on your hologram cloud dashboard under the console menu (click console menu on the left):

This show that you are fully connected to cloud, your account is activated, working and can receive data from your connected devices and sensors. Cool!

You can try and send message to you device from the cloud:

sudo hologram receive --modem nova --sms 

sudo hologram receive --modem nova --data 

The above steps are very important to complete and verify working as they are the foundation of anything else you do send data to the cloud and receiving data from cloud.

From now onward I will focus on the meat of the competition and documents my experiences stemming from the research that I carried out during those periods.

My exploration spans many hardware and software platforms, many evolving ideas, innovations that are constantly changing. Hologram is also a platform that is still constantly changing, if you follow Ben blogs, the forum and use their Github then you will know what I am talking about here. So based on those experiences and the competition requirements as stated below:

1: Gateways back-hauled through cellular (wifi nodes to cellular, BLE nodes to cellular, LoRaWAN nodes to cellular, etc)

2: Asset tracking (bike, vehicle, kid, pet, etc.)

3: Remote controlling (drones, rovers, boats, robots, etc.)

4: Remote monitoring

I finally decided that a framework will be the best way to about addressing those requirements. Also since I am learning by doing then it will be a step-by-step method (Waterfall method).

I went for MQTT as I find it more easy to implement within the time framework. Also I decided to use Mosquitto as my broker ( I love C coding, natural with me).

Mosquitto broker is easy to build, configure and deploy on Linux including Pi-Zero. There are many clients available that could be used as drop-in(s) or customised for your domain. We will be working with both C and Python Clients in this project.

Lets go:

Create a project folder named: hologram, next create a sub folder name mosquitto, download its code from Github, build it and install it.

mkdir hologram
sudo apt-get install build-essential
sudo apt-get install libc-ares-dev uuid-dev libwebsockets-dev libssl-dev 
docbook-xsl xsltproc 
cd hologram
git clone https://github.com/eclipse/mosquitto.git  
cd mosquitto
make

If the above steps are followed then you should be able to compile and install a fresh and latest version of mosquitto. I sometimes find those package with Debian and Ubuntu a bit outdated. More also will be customising the packages for our own specific domain. If all went as said above then your console should resemble mine below:

Next step is installed it on your Pi-Zero, note here that you need the sudo command to be able to do this.

sudo make install

see my screenshot below for an installed version of mine:

Check if its working: type the command below

mosquitto --version 

You get a reply like below:

Error: Unknown option '--version'. 
mosquitto version 1.4.14 (build date 2018-01-05 18:23:48+0000) 
mosquitto is an MQTT v3.1.1/v3.1 broker. 
Usage: mosquitto [-c config_file] [-d] [-h] [-p port] 
-c : specify the broker config file. 
-d : put the broker into the background after starting. 
-h : display this help. 
-p : start the broker listening on the specified port. 
     Not recommended in conjunction with the -c option. 
-v : verbose mode - enable all logging types. This overrides 
     any logging options given in the config file. 
See http://mosquitto.org/ for more information. 

the output window shows mosquitto replying with its version number, build date and applicable options to run it.

Note: You may want to modify the configuration file: config.mk
see: /etc/mosquitto/mosquitto.conf 

I change my country code to en_GB in the config.mk, also I played around with some of its values just to satisfy my curiosity.

To run it issue the the following command: mosquitto 

You get a reply that is running and listen on following ports:

1515177803: mosquitto version 1.4.14 (build date 2018-01-05 18:23:48+0000) starting 
1515177803: Using default config. 
1515177803: Opening ipv4 listen socket on port 1883. 
1515177803: Opening ipv6 listen socket on port 1883. 
^C1515177824: mosquitto version 1.4.14 terminating 

Now that our broker is running, we need the client devices to be set up as well. After that we route clients subscription and publishing to the cloud via mosquitto our broker.

I worked on OwnTrack as a jumpstart for my own framework. See:

https://github.com/owntracks  
and
http://owntracks.org/ 

Now we do this inside the hologram directory that we are working from:

mkdir owntracks
cd owntracks
git clone https://github.com/owntracks/recorder.git 

Note: There are binaries available for Debian and Pi but I am building form source since we will be modifying/adapting for our own framework that collaborate with Hologram-Cloud. What we will have then is as shown below:

Next do the followings:

Copy 
Note: modify it for your local and specific usage

for my initial setting for the core of the framework we are creating, I made the following changes:

# Do you have Lua libraries installed and want the Lua hook integration?
WITH_LUA ?= yes
# Do you want support for the `pingping' monitoring feature?
WITH_PING ?= yes
# Do you want support for removing data via the API? (Dangerous)
WITH_KILL ?= yes
# Do you require support for OwnTracks Greenwich firmware?
WITH_GREENWICH ?= yes

Yes for Lua since we will be doing a lot of scripting in ESP8266 where Lua is native.

Yes for removing data via API ( if you post then update and delete for be ok too!)

Yes for Greenwich firmware. The reason for this, is that the hardware will be use as template and jump-start BSP(Board Supporting Package) for other microcontrollers we will be porting into this framework. You start with what works and adapt it for other embedded devices. The hardware is an ARM MCU. See below:

Its being used as tracker for vehicle anti theft, Can Bus, Accelerometer and many more... We will be using our ESP8266, SensorTile, RaspBerry Pi and many more as replica of this device.

The next part now is getting our local and remote client devices talking to MQTT via recorder. The first one of these devices is being prepared as shown in the video below:

The Robot communicates with mosquitto our MQTT broker and a App was able to pin-point it as well. But we still to connect it to Hologram Cloud which I need to figure out. So the fight is still on.

Using Hologram Kits to do Asset Tracking:

We had two weeks extension given to the deadline so and I wanted to improve upon my submission so that I could cover some areas that I was contemplating to have a go on them.

Asset tracking, Connected Smart Devices, Industrial Automation are some of the sectors that will be using the Internet of Things to enhance their productivity.

Imagine in a situation where a customer walks into a store and start to wandering around thinking about to buy or not to buy or whats is on offer? Tracker App could be develop to inform, improve loyalty between customers and a seller.

Towards this, I created a small App named Hologram-Tracker that works with Beacons, Bluetooth, Esp8266 and could be programmed for proximity, indoor-location and global location. The global aspect of the App does not need a GPS but will need internet connection to relay(Pub/Sub) to Hologram Cloud Server based on MQTT. The method is battery efficient, cost less to nothing to be honest.

See below for screenshots below for it look like.

The App can be trigger by nearby beacons, of course those beacons must be programmed to advertise (may be programmed as peripheral or central). Most will be ready only but in industrial automation when augmented with PLC logic controllers then it be programmed to advertise, read and write. This may also be the case in off-shore monitoring, farming, dairy line or in-vehicle internet of things and supply line production environment.

The App can be downloaded from the the link shown below. Both the source code and .apk (Android for now, ios to follow due to time)

https://github.com/sanyaade-projects/HologramTracker-App

You will find both the source code and the binary there. To modify it, you may need Evothings Studio, Android Studio or Android SDK, Cordova or PhoneGap installed on your systems.

Some of the materials used during the App development sessions

A local Beacon Gateway to cloud used during development:

A video showing the App in operation is listed below:

Links for restiFul or MQTT from my Pi-Zero MQTT broker (code on Github)

https://github.com/sanyaade-projects/Hologram-Tracker

See : https://hologram.io/docs/reference/cloud/http/#/reference/account-management/organization-account-balance/send-message-to-a-device for guide to RetiFul, json and Mqtt.

It has been an exploration and a journey into the world of Internet of Things and Hologram-Nova Kits and the possibilities and features that can be derived from it. Cheri o!

Summary/Conclusion:

An interesting and challenging project that could be deployed in various section of Internet of things that spans different areas from Robotics, Automation, Industrial Tracking, Off-Shore and On-Shore, Automotive monitoring and visualisation to mention but just a few of those expanding and evolving areas of internet of things.

The notion of framework means that resources developed during the competition can be enhance, modify, adapted and personalised for domain specific requirements. I went for a framework so that my effort here can used and not as a stop-gap for this competition.

I will be upload more resources to the Github over the week as I need to trim down and remove object files as per Github restriction on 22MB or less file size.

Interface through API to Hologram-Cloud (got this done as per post on mobilize portal. many thanks!).

Many thanks to you all.

God blesses!!!

Best regards,

Sanyaade