LoRa-Tooth: Small BLE Sensors Over WiFi & LoRa Gateways

Hey, tl;dr?

I get it. Here's a few pictures and short YouTube videos to explain.

Contents

(ctrl-F on "Part #" for navigation)

• Part #1: Motivation
• Part #2: Architecture
• Part #3: Example: BLE door/window open-closed sensor, gateway
• Part #4: Data visualization, alarming
• Part #5: Overcoming range and security issues
• Part #6: Example: Temperature sensor, light sensor
• Part #7: Example: Dog food sensor
• Part #8: Example: Window gap sensor
• Part #9: Example: Geolocation Beacon & Easy Button
• Part #10: System performance
• Part #11: More Examples: DIY buttons and dog-on-couch sensor
• Part #12: Example: Washer/Dryer cycle complete sensor
• Part #13: Conclusion

Part #1: Motivation

Today, we have a lot of tools for building wireless devices. The ESP8266 is probably on the top of the popularity list. It ticks off a few important requirements

• Accessible: really cheap, lots of people can afford the $4 ESP8266 on ebay
• Community: large ecosystem of sensors and peripheral support because of Arduino
• Integration: easy to integrate into other platforms using protocols like MQTT
• Low Complexity: Easy to use IDE, not much tool chain setup
• Hardware exists, and in useful form factors (easy to build with, as long as it's wall-powered)

All of these factors contribute to maximizing "makability". The ESP8266 and other WiFi microcontroller boards make it easy to build IoT devices when size and power consumption isn't a factor.

But today, it's still a big challenge to build practical battery powered sensors. For example, it's still hard to make a DIY open/close sensor for a door or window - one that's small, not ugly, will last for a couple of years on a coin cell battery, and can be built for less than $10. There doesn't exist a platform that checks off the same makability features as the ESP8266 for battery powered projects.

I've seen herculean efforts to make the ESP8266 battery powered, but those approaches typically sacrifice too much (physically too big, still not energy efficient enough, or lose the ease of programming and peripheral support because of switch to IDF and different radio protocol). Other wireless sensor projects use 433MHz or 915MHz transceivers and an ATtiny, but still end up kind of big and not low power enough for what I need.

The popularity of the Amazon Button is evidence of these challenges. It's still hard today to DIY a simple, portable, battery powered IoT button that won't cost more than the $5 Amazon Button. The popularity of the Amazon Button hack, given its very limited functionality, is an indication that we need better platforms for low powered IoT devices.

So many wireless sensor projects need to be small and battery powered in order to be practical, and there's a big technical gap between those requirements and using a WiFi based modules like ESP8266.

Enter the Nordic nRF51822

Let me tell you about this BLE modules that's been flying under the radar of the maker community. It's the nRF51822 from Nordic Semiconductor. For reasons I don't understand, this Nordic BLE module is available from quite a few sellers on eBay and Aliexpress. And it comes in a few useful form factors that make it easy to build with. And it's pretty cheap.

• They come in smaller low pin count modules, as well as larger high pin count modules.
• They all run from 3.6V down to 1.8V directly off Vdd, perfect for the discharge curve of coin cell batteries. No need for external voltage regulators.
• Low sleep current of 4uA.
• These BLE modules are really easy to program with MBED. You don't even need any special hardware, you can use a Raspberry Pi to flash the firmware. More on that later.

That round "Button Beacon" is a nRF51822 BLE module with exposed programming pins, integrated coin cell holder, and an integrated BUTTON! You can buy it and use it as an Amazon button without having to modify any hardware. You just need to program it with the firmware.

The firmware.

I have some ARM MBED projects for these cheap BLE modules. The sensor firmware does things like read I/O, package up the sensor data into a Bluetooth advertisement, and then send those BLE advertisements to a WiFi or LoRa gateway, then to a MQTT broker.

My goal is to architect an "ESP8266-esq" platform for battery powered sensors. It involves a bit more complexity and network architecture than just an ESP8266, but that complexity is isolated to the communications portion of the software that you don't really have to touch. The communications portion is all taken care of, and you just have to modify the sensor portion of the firmware for your own unique sensors. Or just use one of the ones in the project as is.

So, this is another wireless sensor project, but with serious focus on

• small, makable, practical BLE sensors
• low power, for real
• Community Platform & Low Complexity: using mBED, a platform with a large community support for I2C/SPI peripherals, and doesn't require hours spent on setting up IDE and tool chain.
• inexpensive components, so that as many people can make as possible
• flexible wireless architecture to extend BLE to meet long range wireless needs
• integration examples for making useful dashboards and interfacing with other internet services to react to sensors

I'll detail the building process for making a series of sensors using this architecture. It's important that the DIY sensors not have that DIY look. Sensors need to be small, and not tethered to power cords.

Part #2: Architecture

Before jumping into the mechanics of building the sensors, let's describe the architecture and the hardware choice. If we want something that's small and battery powered, we may have to sacrifice wireless range. With Bluetooth sensors, I'm using a couple of ESP8266 WiFi modules to act as gateways. A few of these gateways spread around the house should provide enough coverage to ensure you can see any Bluetooth sensor from at least one WiFi Gateway. With my house, the sensors can pretty reliably transmit through one room between the gateway and sensor, so I only need two gateways. The floor plan below isn't my house, but just an example of how these gateways might look spread around a sea of sensors.

The BLE-WiFi Gateways are composed of 2 parts:

1) a Nordic BLE module that scans for sensor data and outputs the data as JSON string via serial UART.

2) ESP8266 that takes the JSON string as input and iterates through the key-value pairs, publishing this data as MQTT.

Hardware wise, you can use any of the Nordic BLE modules shown previously. In the picture below, I'm using three different BLE modules combined with the ESP8266 to form gateways for the sensors. All three are functionally equivalent. The gateways just need to be plugged into a USB phone charger to power it on all the time.

So you'll need these parts:

• ESP8266 WiFi module [modify MQTT server IP address to your raspberry pi, and gateway friendly name "bedroom", "livingroom", etc]
• nRF51822 BLE module [no changes required]
• Breadboard

The hookup is pretty simple, you just want to power the BLE module from the ESP8266, and wire the serial output from the BLE module to the serial input on the ESP8266.

The code for the ESP8266 is located in my LoRa-Tooth github project, and can be programmed with the Arduino IDE. The code for the BLE portion of the gateway is on mBed. The details on uploading firmware to the BLE module using mBed is given in the next step along with the sensor build.

The wireless BLE data is published as MQTT data on a Raspberry Pi. The general idea of turning wireless sensor data into MQTT is a pretty well established design pattern for IoT projects. Here's a clip of me subscribing to everything on the MQTT broker of my Raspberry Pi and seeing the stream of BLE sensor data.

We need something to provide the visual and processing elements for this MQTT sensor data. This is where Node-RED, OpenHAB, and InfluxDB/Grafana can be used. They can subscribe to MQTT topics and do something - display the data in a nice chart, send an email if sensor value goes above some number, etc.

So that's the general idea. The next step goes into more detail on building the BLE sensor, mBed, and how BLE works.

Part #3: Example: BLE door/window open-closed sensor, gateway

I'll cover more details on the BLE portion of the project in this step. The basic steps are:

1) Buy BLE modules for the sensor and gateway

2) Program BLE modules using mBed online compiler and OpenOCD

3) Put the hardware together.

Shopping

The Nordic nRF51822 modules are all about $5 each. Search eBay or Aliexpress for "nRF51822", and look for modules that look like these. Although any nRF51822 will work (we only need a couple of I/O pins for the a door/window sensor), pick "small and useful" board because it's the smallest and fits nicely in the enclosure I've picked out. You'll need two nRF51822's; one for the sensor and one for the gateway.

While you're at it, buy some of the enclosures and CR2032 holders like these.

You'll also need some "SPDT Magnetic Reed Switch" single pole double throw 3-pin reed switches.

Programming

There are a few ways of loading the firmware onto the BLE modules. Let's just use something you likely already own. It boils down to two simple steps.

1) Generate the firmware file using ARM MBed online compiler.

2) Download the firmware onto the nRF51822 BLE module using OpenOCD from a Raspberry Pi.

One of the cool things about ARM MBED is that it's super easy to replicate someone else's project and generate firmware files. For this example, we'll need to generate the sensor firmware and a BLE gateway firmware. Create an account on ARM MBED and then head to these two projects, hit import, and then select the correct platform and compile. Two easy steps below:

1) Gateway MBED Link

2) Door Sensor MBED Link

Make two BLE modules. One with the sensor firmware, and one with the gateway firmware. Again, you don't have to use the same type of module for the gateway firmware, whatever floats your boat.

Once you have the gateway and sensor firmware saved to your local drive, you can use a Raspberry Pi and an open source application called OpenOCD to load the MBED binary files into two nRF51822's. You just need to connect 4 wires on the BLE module to the GPIO pins on the Raspberry Pi, and run a few commands in the engineering notes below.

The link below are my engineering notes with commands to install OpenOCD, as well as what commands to type in to load the firmware onto nRF51822 BLE modules. If you run into trouble, here's a copy of a Raspberry Pi image that has OpenOCD loaded already. You should have no problems loading the firmware with this image, but use Pi2 or Pi3 (not Pi3+) for this image.

1) Engineering Notes

2) Raspberry Pi Image [careful, it's a 8GB Image, Raspbian Jessie]

If you want to do everything from scratch, follow Adafruit's OpenOCD instructions to compile it on the Pi. Once that's done, move the two BLE binary files onto the Pi and program the modules one at a time.

To use OpenOCD, open up two ssh sessions to the Raspberry Pi with the module wired as mentioned. On one SSH window:

cd /usr/local/share/openocd/scripts/interfacesudo openocd -f interface/raspberrypi2-native.cfg -c "transport select swd; set WORKAREASIZE 0" -f target/nrf51.cfg

On the other SSH window, enter this set of commands, where xxxx.hex is the name of the firmware file. If you use my Raspberry Pi image, SMB is enabled so you can throw binaries directly into the shared folder in /media.

cd /usr/local/share/openocd/scripts
telnet 127.0.0.1 4444
halt
nrf51 mass_erase
flash write_image erase /media/xxxx.hex 0
reset

You'll see this as it flashes. Some of the messages sound like errors, but don't worry that's normal.

In the next few steps, I'll show examples of using other sensors with these BLE modules. But the idea is that the set of sensor MBED examples provide a good set of code for this architecture. If you have your own sensors, you can utilize one of the examples as scaffolding and only need to change a few lines.

Assembly

Once we've programmed the modules, it's time to assemble the sensor. There's not much of a schematic to speak of. We're basically wiring the reed switch to the BLE module. Here's a few pictures to help explain how to assemble it.

BLE To MQTT Conversion

Once that's done, you can test the sensor by monitoring the MQTT topic. Subscribe to all topics and move the magnet around the reed switch.

mosquitto_sub -h localhost -v -t '#'

You'll see something like this:

The MQTT topics for rssi and battery voltage are generated automatically by the sensor example code, using byte locations in the BLE advertisement documented here. Both the sensor firmware and the gateway firmware have agreement on where these bytes are. The actual sensor MQTT topic name ("mag" for magnet presence, "temp" for temperature, etc) exists as JSON within the BLE advertisement, and can be customized easily. The gateways automatically parse the sensor's BLE advertisement for the sensor's JSON data and publishes it as MQTT topic. For example:

BLE[0-17]    = bytes for replay attack, battery voltage, etc...
BLE[18]      = 'm'    //start of sensor JSON, name can be anything you want
BLE[19]      = 'a'
BLE[20]      = 'g'
BLE[21]      = ':'
BLE[22]      = '1'    //sensor value

is translated by the gateway into this MQTT message

/ble/gatewayID/sensor_MAC_addr/mag 1

The gateway pieces together the sensor JSON from the BLE advertisement with the gateway's own hard-coded ID ("livingroom", "basement", "bedroom"), to form a complete MQTT message and payload.

It might seem very byte-inefficient to use JSON within the BLE advertisement to communicate sensor data. The JSON approach opens up the ability to create runtime MQTT topic names and hierarchies using the standard "/" character. This is pretty useful. When you create the sensor firmware, you're defining the meaning of the sensor metric (temperature, light level, humidity, distance, etc). There's no ambiguity there. Being able to define the last part of the sensor MQTT topic name at runtime allows for easy human readable MQTT sensor data. It also adds the ability to generate topic name hierarchies (at runtime) to make sensor data easier to consume. And it also scales well. You could have 2 sensors on one BLE module and 10 sensors on another. There's no need for gateways and sensors to agree ahead of time which bytes correspond to what sensor values, and maintain a separate cross reference. You just beacon out each sensor's data one or two at a time per beacon session. Some of those metrics you might choose to beacon out more frequently than others within the same module. The sensor data is automatically parsed by the gateway and published correctly. The final topic is fully qualified, identifying a particular gateway, BLE mac address, and sensor metric.

Hey, if this MQTT stuff doesn't quite make sense at this point, I think it might make more sense later on with the temperature/humidity example. So no worries. Just flash the firmware for sensors and gateways and things should work.

Current Consumption & Battery Life

That's pretty much it for the sensor assembly. One thing to note is that I'm using 3-pin reed switches instead of the much more common 2-pin reed switch. This is to make the sensor more energy efficient. With a 2 pin reed switch, when the magnet is present and the two pins are shorted, the sensor will always consume a small amount of current equal to the battery voltage divided by the internal pull-up resistor. For a 3V battery and 15kOhm pullup resistor, this may be 200uA, which is 50 times more current than the microcontroller sleep current of 4uA, and would reduce the battery life of the sensor to just one month. The 3-pin SPDT reed switch solves this problem by always providing an interrupt that is an open circuit (not shorted) in both situations (magnet present or magnet absent). The BLE module firmware wakes up when reed switch is activated, and it disables the pull-up resistor on the pin that is shorted and sets the interrupt on the other (open circuit) pin. That way, there is no persistent current flow from the pull-up resistor, and we still wake up immediately when the reed switch is activated.

Double checking the current consumption using a scope, things look correct. The reed sensor is only awake for short amount of time when the switch is activated. We're not wasting a lot of time awake. The sleep current is under 4uA. On a single CR2032 battery, even assuming several open/close events a day, we should have a battery life of over 2 years. Notice that BLE advertisement are very short bursts of less than 2ms of transmit time. The cost of transmitting a BLE advertisement is so small that it's often a good strategy to just advertise more often than to come up with a scheme to acknowledge that the beacon is received by the gateway.

So that's the wake-up and first advertisement. To better ensure the packet is received, the firmware not only advertises on all 3 channels, but also multiple times a second for a couple seconds (feel free to customize amount of repeat). If I zoom out, you'll see each of the peaks below is a single advertisement on all 3 channels. So you see about three advertisements per second. Each repeated advertisement is the same data. The BLE-MQTT gateway hides these repeated advertisements, so only a single MQTT publish is seen per unique advertisement from a sensor. In general, it makes sense that the gateway should hide the cruft that might exist on the wireless protocol.

Part #4: Data visualization, alarming

Once we can get data from a sensor over the air into the Raspberry Pi in the form of MQTT messages, we want to do something with that data.

• Display it on a dashboard (Grafana, OpenHAB)
• Use sensors to turn on lights or adjust other home automation devices (OpenHAB, Home Assistant)
• Send an email or tweet if sensor value meets some criteria (NodeRED)

Home Automation

Let's just pick one of the popular open source home automation platforms like OpenHAB or Home Assistant. I've been a long time OpenHAB user, so I'll use it for the specific examples, but both are similar. They are both programs that can be installed on a Raspberry Pi. They both provide a mechanism of creating dashboards to display MQTT data, which fits very well with what we're doing. But more importantly, they also integrate well with commercial home automation gadgets like Phillips Hue lights or Nest thermostats. Using the scripting system within these platforms, we can subscribe to sensor events, and invoke changes to physical devices like lights and thermostats. Like turn on lights when a door is opened. Or turn down the thermostat when no one is at home (see geolocation step later).

On OpenHAB, you define "items" by giving them names and specifying what MQTT topic that item should be associated to. The wild cards in MQTT provide a lot of flexibility in our distributed gateway architecture. The door or window status can be picked up by any gateway and will update the sensor value. We don't have to specify a specific gateway. This works well with some of the Amazon Buttons we made later that may not be stationary devices. See <demo.items> file in attachments.

Once the items are created in OpenHAB, you can put them into the sitemap and the program provides a webpage as well as a mobile app that displays the dashboard.

Influx + Grafana

To view historical data, as well get get fancier visualizations of data, I like to use Influx Database in conjunction with Grafana front end.

The nice folks over at OpenHAB have a pretty good tutorial for setting up InfluxDB and Grafana. Follow this link here for the setup.

Once that's setup, configure telegraf with the <telegraf.conf> configuration file in the attachment. Telegraf is a high performance MQTT ingestion tool, and it takes the MQTT data and stuffs it into Influx database. Once that's done, we can use the feature rich front end components in Grafana to display sensor data.

Displaying the current status of a door sensor is pretty simple. In Grafana, this is how to get a single state panel to show the open/close status of a door sensor's MQTT topic. Place the MQTT topic corresponding to that BLE sensor in the FROM field.

Grafana also has a useful feature of listing matches as you type. So, if you only record the last 4 digits of the BLE sensor MAC address on the sensor, you can start typing that in, and Grafana automatically populates a pick list of all the metrics from that sensor. Because the sensor data comes in as JSON keys, the pick list is human readable. No need to cross reference address fields. You can also search for all sensors transmitting a particular metric (e.g. all temperature sensors).

You can also get a visualization of the door/window activity using the discrete values panel. It's like a time series chart, but used for metrics that have only a few possible values.

It's a really awesome panel type. It provides a visual of when the door is open or closed, and also calculates the amount of time the door is in the opened state. Pretty neat! The selection of panel types available in Grafana allows for rich dashboards that effectively communicate a variety of sensor data.

I wanted to show the open/close sensor in a video clip, but that video was kind of awkward. Different platforms have different built-in delays. OpenHAB has pretty short delays, but Grafana has much longer delays, about 5-10 seconds. It's not a problem for most dashboard use cases, but for video demonstrations, it's can be a bit awkward. Here's another video clip that uses OpenHAB's dashboard tool to display MQTT data. You can see it's faster to react, and does a better job of showing when the sensor actually transmit data. But that kind of response is rarely needed for dashboards. Grafana provides much more rich data visualization tools.

Part #5: Overcoming range and security issues

There are two main problems with BLE advertisements: security and range.

Security

One of the problems with using BLE is that anyone can see your data and maybe also spoof it by duplicating the BLE advertisements. This is also known as a replay attack. To try to protect or at least detect when spoofing happens, the sensors transmit a byte that represents the number of seconds since the sensor was powered up, with some periodicity. The BLE gateway keeps track of this value, and can validate whether a certain transmission has this correct time signature inside the encrypted data. You can replicate spoofing by removing the battery from a sensor, waiting a few seconds, and then putting it back in and then making the reed switch activate. You'll see something like this come in through the MQTT stream for that sensor.

When spoofing events happen, they come in as a /spoof topic. You can detect spoofing for particular sensor by subscribing to

mosquitto_sub -h localhost -v -t '/+/+/fff7cea88a8b/spoof/#'

Or you can subscribe to any spoofing event by

mosquitto_sub -h localhost -v -t '/+/+/+/spoof/#

Realistically, you wouldn't use mosquitto to systematically capture these events. You'd use Node-RED or pack it into InfluxDB as an event to aggregate (more on this later). The design of the MQTT topic names makes it easy to monitor and alarm against spoofing.

My effort at replay attack prevention is only just OK. Starting conditions and the need to simplify things limits how secure the BLE sensors are. Under this algorithm, it's still possible to spoof sensors if someone captures sensor data and replays it with exactly the same timing four times in a row. Since I need some way to let sensors back into the system after a battery change, I used 3 consecutive advertisements that have consistent time signatures to indicate that this sensor is legit. The screen shot below shows that spoofed sensor eventually let back into the system. The nice feature is that it's easy to detect when this happens and at least be informed.

I have a note with a much longer explanation of the spoof avoidance algorithm in this mBed notebook. I'm not sure if it's a better explanation, but it is longer. The link also has a very complete breakdown of the bytes inside an advertisement.

Range

BLE has very limited wireless range. These particular modules only transmit at 4dBm. Certainly nothing to write home about. To compensate for this limited range, the set of BLE-WiFi gateways act as a distributed network of gateways that use MQTT to aggregate sensor data. It works pretty well for typical home automation situations; the BLE-WiFi Gateway is small enough that you can stuff it into a few places around the house and it won't get in the way or look super ugly.

But what if WiFi doesn't have enough range for a particular use case? For example, let's say you have a farm with a barn that's pretty far away from the main house. You may still need the benefits of low power small form factor Bluetooth module, but have near by mains power to utilize a longer range higher power radio technology to bridge the bluetooth back to a central gateway. In this case, I'm using these RFM95W LoRa modules from HopeRF. They can be brought with an integrated Arduino 32U4 for about $16 each. The data format used for BLE can be replicated with LoRa to make a BLE-to-LoRa bridge that will extend the range of BLE. We just need new code for a LoRa-MQTT gateway and the BLE-LoRa "bridge".

I happen to be using an Adafruit version of the LoRa 32U4 module, but I also have the generic Aliexpress/ebay version and I know these cheaper versions of the 32U4 works just like the Adafruit one, complete with LiPo battery charging circuit and all.

Since the BLE-LoRa bridge is using Arduino hardware, you can use the Arduino IDE to program the 32U4 modules.

1) BLE-LoRa Bridge [Github Link]

For the BLE module, use the same MBed BLE gateway code previously mentioned. It's just the Arduino code for the 32U4 that is new.

2) LoRa-MQTT Gateway [Github Link]

The LoRa-MQTT gateway works much like the BLE-MQTT gateway does, but this time just replacing the BLE module with an Arduino and a raw RFM95W LoRa radio. The Arduino in this case is a Pro-mini clone. The code for the gateway also works if you just want to replace the Pro-mini and RFM95W with the 32U4 LoRa module.

With these LoRa modules, the range between the BLE-LoRa bridge and the MQTT gateway can be as much as 1500ft without having to do anything special (just wire antenna placed at table height). I used this GPS LoRa sketch along with the 32U4 LoRa module to test range.

Data from BLE modules coming in through the LoRa bridge show up with a /yyy/ topic name instead of a /ble/ topic name, but otherwise, everything works the same. It's a slight issue with Grafana because Grafana doesn't understand the idea of wildcards in MQTT topic names. Using the Node-RED flow in Step 8, I actually remove the specifics of where the sensor data comes from, and generate a generic MQTT sensor topic so that it's easier to consume that data in Grafana. In other platforms like OpenHAB, you can use wildcards to ignore the first level that indicates ble or LoRa, sensor data is easier to display.

Part #6: Example: Temperature sensor, light sensor

Now that we've covered the details of downloading firmware and understand just how awesomely simple it is to use MBed for firmware generation, we can go on to build more sensors without having to cover every little step.

Temperature sensors that work over I2C are pretty useful. Two of these common sensors are the TMP102 and the Si7021 (which also has humidity sensing). They're only a few dollars on ebay/aliexpress. They both run in the same voltage range as the nRF51822, and all that's needed is to hook up the I2C lines.

Depending on the alignment of the SCL and SDA pins on the sensor to the BLE module, modify code so you don't have to physically cross the wires to hook up the I2C pins.

The MBed project to compile is listed here:

1) TMP102

2) Si7021 w/ light sensor

Here's what the TMP102 sensor build looks like.

Here's how I built the Si7021 temperature and humidity sensor. One of them is just the Si7021, the other one adds a light sensor and voltage dividing resistor to sense light level. By the way, if you're not seeing more than one image below, Hackster uses the horizontal scroll arrows to make it easier to group lots of images without taking up vertical scroll space.

One interesting thing I learned about the digital temperature sensors is that they take different amounts of time to make sensor readings. The TMP102 only takes about 4ms. The Si7021 takes 21ms. See scope image below. Those seem like pretty big differences, but compared to total battery life, it doesn't make a lot of difference. At 10 minute transmit intervals, I think the Si7021 can still provide 2 years of battery life though.

Because the battery voltage is part of the BLE advertisement data, we can use Grafana to help us monitor the battery status of the sensors. See the chart below. One of the kid's room temperature sensors is a Si7021 and the other is a TMP102. You can see they behave slightly differently (more jigger on one). I've been doing a battery life test by having the sensors take readings and sending out temperature data every 30 seconds (green) and 15 seconds (orange). They've been running this battery test since last October (7 months ago), and the battery voltages are still at 2.68V and 2.93V. In normal conditions, you probably won't need temperature data so frequently, and I expect a single CR2032 should last more than 2 years.

The cool thing about collecting temperature data and saving it to InfluxDB is that you can make charts and start seeing patterns. During a winter day, this is what we see for one of the bedrooms.

Besides seeing the temperature go up and down with room activity, you can also see the temperature ripple from the furnace cycling on/off about twice an hour. Pretty cool!

There are lots of things you can build with sensors. The set of MBed code in this project provides a scaffolding of code to make it easy to change a few things specific to the sensor or your particular use case. But these changes can be kept pretty small and still enable you to take advantage of the rest of the features in the architecture.

Part #7: Example: Dog food sensor

One of the problems my family runs into is that we sometimes forget if we fed the dog. You start asking yourself "I remember feeding the dog, but was that yesterday or today?" And Cody has kind of figured out that if he just acts hungry, he has a chance of getting two breakfasts.

A way to solve this problem is to just put a BLE module with a tilt switch to sense when the food container is opened. While we're at it, we can also add some other sensors to provide more information.

The VL6180 distance module in this dog food sensor is an I2C time of flight sensor that measure the centimeters distance to object in front of the sensor. The sensor is very low power, but also has limited range. It can sense about 20cm. It's not quite far enough to the bottom of the dog food container, but it's close enough. You can find it on ebay/Aliexpress for about $6 each.

I'm using the same nRF51822 BLE module as with the other sensors, but with this slightly modified mBed code to take readings from the distance sensor:

Dog Food Sensor [MBED]

Hardware wise, the major difference between this build and the temperature sensors is that I'm using a 2.8V MCP1702 low current regulator. The distance sensor requires a regulated 2.8V source, so it makes sense to use two 3V batteries in series to get 6V input to the regulator. That way, we can fully drain the battery down before the regulator input voltage is too low to power the regulator. See example circuit above for the regulator wiring.

The dog food sensor uses a tilt switch to wake the sensor from sleep. It then waits until the tilt switch is opened up again (indicating the top is placed back onto the food container), and wait some more time before taking a reading and transmitting the distance. So the sensor's BLE advertisement (coverted to MQTT message) is just </+/+/+/dis>, and the message payload is the cm value for distance.

I'm using Node-RED to scale the sensor's cm value into percent dog food. The Node-RED flow is in the attachments section.

Then I display the value of the percent as well as the time since it was last received. This is done with the single Grafana panels. I really like how Grafana makes this so easy. The Grafana panel interprets this transmission as a feeding, and keeps track of the time since the last transmission. I'm showing the Grafana configuration below.

Part #8: Example: Window gap sensor

With the open/close sensors, we can only tell if a window is closed or not closed. But sometimes we want to know if it's wide open or just slightly opened. I often leave windows slightly opened if it's raining because I like the smell, but I don't want the windows opened so far that rain gets in. So the distance sensor from the dog food container in the last step is perfect for this.

The mbed program for the window sensor is here:

BLE_Sensor_VL6180_window [mBED]

The circuit is the same as the dog food sensor, but with a different orientation for the sensor and different reporting frequencies in the firmware. The images below show how it's put together.

I'm using the tilt sensor to increase the sensor reading and transmit frequency for a short period of time. So, if you open/close the windows with much enthusiasm, the tilt switch will wake the BLE module and the dashboard will display the distance right away. But the sensor also takes regular readings every 10 minutes if the tilt sensor isn't activated, so the dashboard gets updated eventually.

Realistically, I would probably just show the single stat for "closed, gap, opened" rather than the centimeter measurement, but I'm showing both in the demo so you get a sense of how it works. The relevant Grafana settings are in the screenshots below.

Here's a quick video of the window sensor with a ruler to show how accurate it is. It seems to be within 1 cm across the range. The dashboard is an OpenHAB dashboard that's just subscribing to the distance MQTT topic.

Part #9: Example: Geolocation Beacon & Easy Button

The architecture of distributed BLE and LoRa gateways in the project also allow for obtaining pretty detailed geolocation data. For example, you can find out what rooms or areas of the house a dog or cat hangs out in the most. You can use a video camera for something like that, but it's not the same as getting a graph that breaks down how long in each room they hang out in, and how often they move about, and giving a visual sense of their day.

Because of the limited range of the BLE sensors, we're using several BLE-MQTT gateways to ensure we capture the data. Each BLE-MQTT gateway is reporting the RSSI of the sensor data it receives. We can use Node-RED to gather the highest RSSI values, and publish that back as MQTT to be displayed as a "Discrete" panel in Grafana. The discrete chart panel in Grafana is great. It automatically tells you how much time the dog/cat spends near each gateway, and gives a visual sense of how much they move around (probably not much). This can be useful for other purposes too, I'm just using pets as an example. The the chart below, you can see I took Cody out for a walk at around 9:20PM.

We'll start with the actual hardware and move on to the software. The best BLE module to use is this "nRF51822 Beacon" module. They're about $6 and come with a handy plastic case that can be attached to a keychain or a dog/cat collar.

They're a bit more awkward to program, but the same method can be used as step 2, you just might need some clips. See example below.

The mbed program is here:

BLE_Sensor_Button

The program transmits a beacon every 30 seconds. BLE advertisements are short and consume very little power. Even advertising once every 30 seconds, the battery life for the beacon is probably over two years. Since there's also a button on this beacon, the above firmware also implements code for a long press and short press that causes the BLE module to transmit a value (0 or 1 depending on how long you hold the button). More on button usage later, for now, here's how the geolocation works.

This is what the MQTT stream of RSSI values looks like.

That's the hardware and firmware. Now we use Node-RED to keep track of which gateway the beacon is close to. The entire Node-RED flow is in the text file in the attachments section. Here's what the flow looks like:

The flow above uses the fact that gateway location is a friendly name for BLE-MQTT wifi gateways. We subscribe to a particular

/ble/+/XXYYZZ/rssi

topic and parse the friendly gateway name and stores it off. Every 5 minutes, it publishes the gateway name to a location topic for that XXYYZZ mac address.

/g/g/f2a2df72de8a/location

Here's the Node-RED node publishing the locations every 5 minutes.

On Grafana, we just need to display that topic name and the payload which is a string representing the gateway name. Grafana automatically takes care of parsing out the strings and grouping the values. In the event that none of the RSSI values are above the threshold set in Node-RED, the initial condition of "Nowhere" as the location string gets published. That's how you can tell the dog wasn't near the house and is probably out for a walk.

This geolocation of BLE beacons works for all the bluetooth sensors in this project. It's just that doors and windows and dog food containers don't move around, so there's not much point in charting their locations. But in our hypothetical situation of using LoRa to bridge BLE sensors at a barn that's far away from a farm house, this geolocation still works. So, for example, we can track if a farm cat is in the house somewhere or at the barn, or if it hasn't been seen in either location for a while and we should start to worry.

Other Uses

Aside from being able to track a shy cat, there's other potential applications for reliable location data. For example, you can stick the $5 beacon under your bike seat, and set Node-RED to email you if the bike leaves the garage at odd times. Or increase the Beacon frequency to 10 seconds and maybe use it to automate turning on lights or unlocking doors when you (or your car) comes home. I've seen people use wifi or apps on their smart phones to do this kind of geolocation, but the use of beacons is a more reliable approach than mobile apps that have variable responsiveness when running in the background. Home automation platforms like OpenHAB and Home Assistant excel at integration with commercial devices, and they also are able to talk MQTT. Being able to program your own sensors and beacons and do geolocation makes the entire home automation ecosystem much more capable.

"Amazon Button"

The button on the beacon is pretty useful. The plastic case that comes with it is flexible, and pressing on it will cause the case to flex in and depress the button. It's pretty much like an Amazon Button that you can use with Node-RED to interface with other services. You can long press or short press the button to send different messages to Twitter or send email. You can stick it in the kid's bedroom so they have a button to press to ask for help - by playing an audio clip from a Raspberry Pi in the living room for example. Since it's MQTT, any home automation system that can talk MQTT, like OpenHAB or Home Assistant, can use this button to turn lights on/off, play music from Sonos, etc.

Long Press / Short Press on the button publishes different values to MQTT broker

/ble/gateway/mac/BEA 1

/ble/gateway/mac/BEA 0

Part #10: System performance

It's sometimes hard to tell how well an architecture will perform until it's been built and tested. The performance metrics I'm interested in are:

• 1) Reliability - are we able to capture all the sensor beacon data into the database?
• 2) Device Health - can we detect when a device stops transmitting?
• 3) Battery life - how long can sensors last on battery power?

Reliability

There's a few things that can go wrong to cause us to miss data after a sensor transmits it. This isn't an exhaustive list, but they're potential problems I wondered about at the beginning.

1) Signal Lost: BLE signal from sensor never makes it to any of the gateway nRF51822 module. Gateway could just be too far away, or there was signal interference at the time the sensor was transmitting.

2) MQTT Publish Failure: The ESP8266 fails to publish the sensor data to Mosquitto MQTT broker. It's working over WiFi, blips can happen. I'm still using QoS 0 for the ESP8266's MQTT client, so I'm not guaranteeing successful publish events. This is an architectural issue.

3) Ingestion Performance: Telegraph might not be fast enough to receive and write every MQTT published message to database given multiple gateways pushing data simultaneously. This is an architectural issue.

Each BLE beacon includes a byte for "count" that increments with every new transmit. I can use the count value to detect missed data. I haven't automated a way to calculate this, but I can download the transmit count value, and look for non-consecutive counts in the database. Visually, it looks like this when count is charted in Grafana.

The slope represents how often the different sensors are transmitting. I should expect to see regularly spaced points incrementing by one if I zoom way in.

If I look at the count data from a sensor that's far away from the gateway, I see something like this. There are gaps in the count value, indicating some transmissions weren't captured by that gateway.

After downloading the data from InfluxDB and counting up the missed transmissions, I get these results for four sensors close to their gateway. Out of curiosity, I also repeated the exercise for a sensor with a far away gateway.

Sensors in the same room have no problems being seen by the gateway and the data makes its way through to the database. The 522E sensor has a miss percentage of almost 1%, so that might be concerning if it was a door/window sensor that I'm relying on to do alarming. Performance degrades fast when I'm trying to make it across the house (2 rooms away).

The mBED programs for the sensors have somewhat arbitrary advertisement lengths and frequencies. For things like buttons and reed switch sensor, it would make sense to extend the advertisement period and increase the advertisement frequency (that is, how many advertisements per second and for how long to advertise). This would improve the chances that a gateway will see the BLE beacon.

Tic_Stop_Adv.attach(stop_adv_Callback, 3);  //increase 3 to 4
//decrease this from 900 to say....500 to make more advertisement ticks
ble.gap().setAdvertisingInterval(900);

It would also help to place the gateway at a higher elevation. Worst case, I might have to add another gateway to ensure good coverage. Antenna orientation and placement might also be important. I tried to make the sensors so that the battery is not going to be in between the antenna and the gateway. That's something you may want to pay attention to when making sensors.

The important thing to get out of this is that the weakest link seems to be Bluetooth reception, which can be solved with gateway placement or adding gateways. So far, I don't see any issues with the architecture itself.

Device Health-Heartbeat

The mBed example code for all the sensors transmit a heartbeat every X minutes. For temperature sensors, the heartbeat basically defines how often you want the temperature/humidity data.

const uint16_t Periodic_Update_Seconds = 900;

For door/window sensors, the heartbeat provides the same data about the magnet position as when the sensor is activated, but I want to differentiate between a heartbeat and actual door/window activity. To do this, every heartbeat transmission has a '/p' added to the end of the JSON for that sensor value. So, when a magnetic reed switch sensor sees activity (due to pin level interrupt from actual door/window activity), the MQTT topic name is this:

/ble/gateway/XXYYZZ/mag 0

/ble/gateway/XXYYZZ/mag 1

That same sensor will also periodically report the magnet position every X seconds, even if there was no door activity. And these periodic heartbeats come in as

/ble/gateway/XXYYZZ/mag/p 0

/ble/gateway/XXYYZZ/mag/p 1

By subscribing to /ble/gateway/XXYYZZ/mag/#, you'll get the latest position.

By subscribing to /ble/gateway/XXYYZZ/mag, you'll get only activities.

And you can monitor the health of a sensor to ensure it's reporting at least every X heartbeat seconds by subscribing to /ble/gateway/XXYYZZ/mag/#

Battery Life

Here's a summary of estimated battery life for the different sensors I've built. I'm using Digi's battery life excel sheet and modified it a bit for my own purpose. The data captures the measured sleep current, wake current, and wake duration, as well as some assumptions about how frequently I expect these sensors to read and transmit data. I'm pretty conservative, assuming a 240mAh rated coin cell can provide 70% of the rated capacity. Most of the sensors come out to more than 2 years battery life, with the exception of the dog beacon where I'm assuming it transmits every 30 seconds for geolocation purposes. If I change the assumption of 30 second beacons to 60 second beacons, the battery life changes from 1.8 years to 2.5 years. As a gut check, my months long real-world testing doesn't contradict these estimates.

Part #11: Examples: DIY buttons, dog-on-couch sensor

It's immensely powerful to to have a selection of BLE microcontroller boards that you can wire up your own sensors to and be able to easily modify code to read that I/O. Reading I/O is relatively easy, and it's what enables the creation of lots of different applications that all use same general structure.

So, for example, you can take a kids toy that has lots of buttons and make a very low powered button gadget that does whatever you want. Each button can send a unique MQTT message, and from there, Node-RED or OpenHAB or Home Assistant can be used to

• send text messages or emails for help
• turn lights on/off
• start playing music on Sonos speakers
• increase/decrease setpoint on Nest thermostat
• ...whatever else you have your home automation platform connected to

Here's some examples of using the buttons and motion sensing purple BLE module to do useful things.

The purple nRF51822 BLE is a pretty neat piece of hardware. Besides being breadboard friendly, it has lots of pins for I/O, and embeds a low power I2C LIS3DH accelerometer. I modified an existing mBED library to enable motion sensing interrupts on this accelerometer. So the example code for the toy has the microcontroller asleep and wake on motion or button press, and transmit data on which button was pressed or if the toy was bumped. The motion sensing can be extremely sensitive.

A coincell should last quite a while (a year?) depending on how it's used and if you need different timing options. The program re-enables motion interrupts every 20 seconds.

So, just like all the other examples, here's the mBED code and some pictures of how I built the toy.

Puppy Toy mBED Example Code

I started by identifying the the buttons and the corresponding conductor colors. Then I wire one side of the button to the I/O pins according to the table below, and the other conductor to ground (interrupt pins are pulled up).

Tested out the sleep current consumption to make sure I can get optimal battery life. It's about 8uA on sleep.

Here's a clip of the motion sensing and MQTT messages that get published.

Example Node-RED and OpenHAB configuration to respond to the buttons are in the attachments section. But this is really going to depend on what you have in your house and what exactly you want to do with the buttons. You can use OpenHAB or Node-RED to play audio clips when a MQTT topic gets posted (like the air horn in this example). See the OpenHAB configuration in attachments for this example.

Part #12: Example: Washer/Dryer cycle complete sensor

Building up an arsenal of low-power sensors enables a lot more applications. The ability to sense motion/vibration at microamp levels can be used to detect things like washer/dryer cycles, for example. Since the sensor is embedded in the purple BLE module, there's almost no wiring needed other than powering the module with a coin cell battery.

The video demo isn't very exciting because there's a built-in 1 minute delay before things transition to the "running" state. But I wanted to show how sensitive the interrupt threshold on the LIS3DH can be set. It's actually being set off during the water fill portion of the wash cycle. The agitation part hasn't even started.

Here's the link for the mBED code for washer dryer sensing. The modifications are mostly timing related so that slight pauses in the cycle aren't interpreted as completion and vibration sensed while loading/unloading the machine isn't interpreted as machine running state.

[MBED] BLE Sensor Good Vibrations

The program reports run/idle status of the machine like this. 0 means entered idle state and 1 means entered run state.

/+/gateway/mac/vib 0

/+/gateway/mac/vib 1

So we just create items that subscribe to these MQTT messages and display it on OpenHAB. Here's the OpenHAB configuration.

//sitemap--------------------------------------
sitemap laundry label="Basement"
{
       Frame label="Laundry"
       {
               Text item=itm_washer
               Text item=itm_dryer
       }
}
//item
Number itm_washer "Washer Status [MAP(laundry.map):%s]" <mywashmachine> (ALL) {mqtt="<[mymosquitto:/+/+/c8dbdb869c92/vib:state:default]"}
Number itm_dryer "Dryer Status [MAP(laundry.map):%s]" (ALL) {mqtt="<[mymosquitto:/+/+/macaddress/vib:state:default]"}
//transform:  laundry.map 
1=Running
0=Idle
NULL=unknown
-=unknown

Getting sensor data into the Raspberry Pi in a form that's easy to consume (like MQTT) is most of the work. Now we can use the capabilities provided by awesome open source platforms like Node-RED, OpenHAB and Home Assistant to do interactive things like

• Play an audio alarm in the living room on another Raspberry Pi when washer/dryer completes (same method as the air horn noise from the dog-on-couch sensor)
• Play a clip on a Sonos speaker
• Or maybe use Alexa to play the audio.
• Send a text message to your phone
• ...any of the dozens of other integrations available through these platforms.

Once you have a method for getting the data into a Raspberry Pi, the number of options expand quite a bit.

Part #13: Conclusion

Sometimes it's easy to get lost in the details and lose track of the purpose. Being able to make your own gadgets is fun. Being able to do it with minimal fuss and minimal expense is awesome. I really just want to maximize makability and manage the complexity of IoT projects. It's possible to make this project work without fully understanding every little bit at first, and that's a good way to learn.

This Bluetooth pet location beacon doubles as a voice activated light switch. The wake-up-word for mine happens to be "Cody".

There's a ton of other things that can be built. I realize that on a good day, you can probably buy a Zigbee door sensor for $10, and just pair it with one of the off-the-shelf home automation hubs. Why go through the work of building your own sensors?

This project enables people of different skill levels to build nice looking, untethered, one-off sensors to solve problems that can't be solved with off-the-shelf home automation products. And do so without having to deal with horrible development tools that plague so much of embedded systems development.

The Bluetooth Beacon with the built-in button is pretty handy. You can subscribe to the long/short presses and use Node-RED to do pretty much anything - send a text message for help, send an email, turn on lights, etc...

Back when I had my first kid and we were just returning home from the hospital, it seem like the most important thing was to keep track of wet and dirty diapers. I wish I had that button back then. Long press for dirty diaper, short press for wet diaper.

And I would want to make sure to archive the diaper changing schedule as a CSV, in case it turns out to be useful in the future.