Sentinel

Motivation

I created Sentinel because I wanted to know how my dog Duke was doing while being away from home, I have a couple of IP cameras, their setup is complicated, they are expensive and the yearly cost of having an app is high, they are not secure (they can get hacked) and they are static and Duke moves around from room to room so there was no way to follow him around.

I'm an Android Developer so I'm very fond of Java, Google tech and services and most recently Kotlin, so I wanted to challenge myself to create something that combined as much of Google technologies and services as possible, Android Things, Firebase, Play Services, Google Cloud, Vision API, TensorFlow, App Engine, etc. A project like Sentinel also serves as a way to get other people excited about coding, IoT and STEM.

When finalized, Sentinel will be able to patrol around in a semi-autonomous way, find and follow Duke (my dog), recognize familiar faces using TensorFlow, etc. I'm still working on live video and two-way audio via Web RTC.

Sentinel's First Roll

Sentinel's First Play with Duke

Companion Android App

The existing companion Android app allows me to remote control Sentinel and move it from room to room so I can keep an eye on what is happening at home. The app also sends me alerts about movements and loud sounds and helps me keep an Image History.

Android Things as Operating System

I'm using the latest release of Android Things.

Android Text-To-Speech

Look who's talking now! Yes, Sentinel can speak! It can tell you if it is "up and alert", its IP address and its volume level.

Since the Raspberry Pi 3 with Android things supports Android's TTS Engine out of the box, it was super easy to integrate it and let Sentinel be heard trough cheap USB stereo speakers. And let me tell you... when you don't have a display, having your IoT project actually tell you what is happening (when debugging) really helps!

Adafruit Motor Hat

To drive Adafruit's Motor Hat, I'm using an amazing library Antonio Zugaldia translated to Java from python for his Android Robocar project.

When you stack raspberry pi hats, controlled via I2C like I did, you need to be able to address them, for that, I made a slight change to Antonio's PWM and Motor Hat classes, that way I can address the PWM motor hat with a dynamic I2C address and stack as much hats as I need.

UPDATE: I've ported Antonio's Pwm and DC Motor Hat library to Kotlin. They will be available on GitHub as independent gradle libraries soon.

Adafruit Servo Hat

To control the servos that turn the camera pan/tilt bracket up/down and left/right, I'm got an Adafruit's Servo Hat. I've written a Gradle library in Java (soon to become a Kotlin library) that is available on Github so you can use it as well for your projects.

UPDATE: I've ported my Servo Hat library to Kotlin. Will be available on GitHub as an independent gradle library soon.

3D Printing

I found the 3D model for the pan/tilt bracket mount (where the camera is mounted) online and sent it to Shapeways for printing, they sent it via mail 3 days later. Their website is super easy to use, basically you just have to create an account, upload your 3D model(s), pick a material (I picked the cheapest possible, PLA) and order it.

Bluetooth Controller

To control Sentinel wirelessly, I'm using the 8Bitdo NES 30 Pro controller connected via bluetooth. Again, I got inspiration and borrowed from Antonio's Robocar project, he is using a controller from 8Bitdo as well, but I've taken it to the next level adding support for ALL the buttons and joysticks on the NES 30 Pro!. More detail on the setup and customization of the controller will follow and of course, I will put the code on Github so you can use the NES 30 Pro controller on your projects as well. ;)

On 8Bitdo's support page, you will find the latest controller firmware updates and an Android APK you'll need to map your controller buttons.

Firebase Authentication

One of the objectives was to create a secure home security system only I could access, so I'm using Firebase Auth on the companion Android app and Sentinel to authenticate with Firebase so only I can see the images, videos and alerts.

Firebase Real Time Database for Remote Control Commands

I'm using Firebase real time database to send commands to the Cloud from a the companion Android app. Sentinel has listeners that trigger when the command records on Firebase database change and executes the commands right away.

Firebase Storage

I'm uploading the pictures Sentinel takes to Firebase Storage.

Firebase Cloud Functions

I have 2 simple Firebase Cloud Functions:

• generateThumnail: creates a thumbnail when a picture is uploaded to Firebase Storage. The thumbnails are shown on the companion app Image History.

• sendAlertNotification: this function is a Firebase Database trigger that sends me a notification when a new loud sound or movement alert record is created by Sentinel.

Firebase/Fabric Crashlytics

I want to know about errors and crashes on my robot so I'm using Crashlytics to get error reports.

Google Cloud Vision

When a picture is taken it gets annotations from Google Cloud Vision API.

Links: Android Things, TTS Engine, Firebase, Vision API, TensorFlow, Kotlin.