In this project you will learn how to use Azure IoT Edge by Microsoft on a Raspberry Pi Model 3 B+ as the edge device with a Sense HAT attached to it. This project can be used as a base for larger Azure IoT Edge projects or as a Proof of Concept for Azure IoT Edge with real sensors instead of the simulator examples provided by Microsoft (or to just to have fun).
Azure IoT EdgeAzure IoT Suite has great many products in the Microsoft Azure cloud
- to collect data from millions of devices spread out the earth (IoT Hub, Event Hub, Cosmos DB)
- to analyze data collected both in real-time and batch in the cloud (Stream Analytics, HDInsight, Data Lake Analytics)
- to visualize the data on the fly or in rest (Power BI)
- to take actions on the results of the analysis (Logic Apps, Microsoft Flow)
One consideration when collecting data is that it is not always you want all the data generated by the devices in the cloud because of bandwidth, connectivity or even privacy issues. For this purpose Microsoft has introduced the Azure IoT Edge to move some of the analysis from the cloud to the devices and collect only the results of the edge analysis in the cloud instead of all the raw data generated by the devices. And let some of the analysis logic lie on the device.
Before you beginThis project is implemented using Raspberry Pi Model 3 B+ but it might work with older versions of Raspberry Pi as well (no guarantees)
We are using a fresh installation of Raspbian as the operating system on the Raspberry Pi but it might work with a not so fresh installation of Raspbian or any other linux operating system (no guarantees here neither). You can follow the Raspbian installation guide using NOOBS here.
As this project uses Microsoft Azure components it is required that you have an Azure subscription. Although Azure IoT Edge is open source it is a much complete experience if you can see with your own eyes what is possible to do from the cloud. If you do not yet have a subscription in Azure you can create a free account and have $200 credits to use for 12 months.
Sense HAT is used as the sensor board in this project because of its ease of use and official python library. Any other sensor extension board might or might not work. But make sure they have a python library before you chose anything else.
And be very sure you have an officially supported adapter as the new model of Raspberry Pi with a Sense HAT on it consumes more power than many USB power adapters support.
Now we have all the hardware and software in place and ready to start.
Many of the installations we are making on the Raspberry Pi will be done remotely. I am using Putty as the ssh utility to connect to the Raspberry Pi shell. For this to work you would need to enable remote connectivity on the Pi. You can do that by opening a terminal window and running the config utility using the command:
pi@raspberrypi:~ $ sudo raspi-config
Then select interfacing options
→ ssh
to enable it.
When you have the terminal already open you can issue the ifconfig
command to see the ip address of the Pi.
pi@raspberrypi:~ $ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.128.2.80 netmask 255.255.255.0 broadcast 10.128.2.255
inet6 fe80::3bd4:f98:2be9:ed9a prefixlen 64 scopeid 0x20<link>
ether b8:27:eb:9e:ef:8f txqueuelen 1000 (Ethernet)
RX packets 3427 bytes 474575 (463.4 KiB)
RX errors 0 dropped 46 overruns 0 frame 0
TX packets 148 bytes 19534 (19.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
wlan0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether b8:27:eb:cb:ba:da txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
We will connect to this ip address in Putty to connect to the Pi. You can always issue the same commands on the Pi terminal as well. If you have not changed the default password you can use pi
as the login name and raspberry
as the password.
IoT Edge runtime, we will be installing on Raspberry Pi, will communicate to an IoT Hub in Azure. This IoT Hub needs to be created and initialized in Azure platform as well as other resources we will use in this project. It is possible to create these resources on the Azure portal or through command line tool package, Azure CLI, provided by Microsoft. Here we will use Azure CLI for this purpose. It is even possible to run Azure CLI commands in the Azure Cloud Shell. In this project we will use Windows PowerShell on a Windows development computer but Azure CLI works even on macOS and Linux.
1. Begin by installing the IoT extensions to Azure CLI
az extension add --name azure-cli-iot-ext
2. You need to login to your Azure subscription before going forward. You can do that on the command shell using the following command. It will ask you to open a browser and enter the provided code to login. It will even output the list of subscriptions you have after successful login.
az login
3. If you have multiple subscriptions you can see a list of them and which one is the default subscription. You can even change the default subscription. It is important to know which one it is if you have multiple subscriptions. Because all the resource you create in Azure will be created in this subscription.
az account list
az account set --subscription "<SUBSCRIPTION NAME>"
4. Create a resource group in Azure to put all resource you will create into. Here I have used the IoTEdgeResource
as resource group name and northeurope
as data center location. You may pick anything that you like.
az group create --name IoTEdgeResources --location northeurope
5. Create a new IoT Hub in the resource group. Note that the hub name should be unique in all Azure. I am using IoTEdgeHubHelloWorld
in this example (which you can no longer use as this name is occupied by me now.) And the pricing and scale tier I have selected for the hub is S1 Standard
. You can select F1
for a free tier but you may have at most one free tier IoT Hub per subscription.
az iot hub create --resource-group IoTEdgeResources --name IoTEdgeHubHelloWorld --sku S1
6. Create a new edge enabled device to the newly created IoT Hub. The name I am giving to the device is myEdgeDevice
az iot hub device-identity create --hub-name IoTEdgeHubHelloWorld --device-id myEdgeDevice --edge-enabled
7. Retrieve the connection string for the device which we will use later on.
az iot hub device-identity show-connection-string --device-id myEdgeDevice --hub-name IoTEdgeHubHelloWorld
Copy the connection string part from the output of the previous command to be used later on.
{
"cs": "HostName=IoTEdgeHubHelloWorld.azure-devices.net;DeviceId=myEdgeDevice;SharedAccessKey=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX="
}
Install IoT Edge on Raspberry PiNext step is to install the IoT Edge runtime environment on Raspberry Pi. As Raspberry Pi runs on ARM architecture we need to install that version of IoT Edge on Raspberry Pi. Open a terminal on Raspberry or remote connect using ssh or Putty. IOT Edge runtime is a container based environment and uses Moby engine which supports Open Containers Initiative format.
1. Download and install moby engine
curl -L https://aka.ms/moby-engine-armhf-latest -o moby_engine.deb && sudo dpkg -i ./moby_engine.deb
2. Download and install moby command line
curl -L https://aka.ms/moby-cli-armhf-latest -o moby_cli.deb && sudo dpkg -i ./moby_cli.deb
3. Fix the installation
sudo apt-get install -f
4. Install security manager implementation for IOT Edge
curl -L https://aka.ms/libiothsm-std-linux-armhf-latest -o libiothsm-std.deb && sudo dpkg -i ./libiothsm-std.deb
5. Download and install the IOT Edge Security Daemon
curl -L https://aka.ms/iotedged-linux-armhf-latest -o iotedge.deb && sudo dpkg -i ./iotedge.deb
6. Fix the installation again
sudo apt-get install -f
7. Setup the device to connect to the right IOT Hub. Open the configuration file using nano editor on the Pi.
sudo nano /etc/iotedge/config.yaml
Go to the field where the manual connection string is provided and paste the connection string you retrieved from Azure in the previous section.
###############################################################################
# Provisioning mode and settings
###############################################################################
#
# Configures the identity provisioning mode of the daemon.
#
# Supported modes:
# manual - using an iothub connection string
# dps - using dps for provisioning
#
###############################################################################
provisioning:
source: "manual"
device_connection_string: "HostName=IoTEdgeHubHelloWorld.azure-devices.net;DeviceId=myEdgeDevice;SharedAccessKey=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX="
Save and exit using Ctrl-X,Y,Enter
.
8. Restart the iot edge runtime for the changes to take affect. The runtime will be started automatically if you restart your PI for any reason.
sudo systemctl restart iotedge
And you can at anytime list the modules running in IOT Edge and see their status.
sudo iotedge list
As we have not installed any additional modules yet so the list will be quite short at this moment. Only edgeAgent
is available.
NAME STATUS DESCRIPTION CONFIG
edgeAgent running Up 2 minutes mcr.microsoft.com/azureiotedge-agent:1.0
And to see the status of the IOT Edge runtime environment as a whole you can use another command.
systemctl status iotedge
This command will show you current status, the CPU time in total and the time the daemon was last restarted.
● iotedge.service - Azure IoT Edge daemon
Loaded: loaded (/lib/systemd/system/iotedge.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2018-11-29 09:56:58 GMT; 3min 19s ago
Docs: man:iotedged(8)
Main PID: 3627 (iotedged)
CPU: 28.178s
CGroup: /system.slice/iotedge.service
└─3627 /usr/bin/iotedged -c /etc/iotedge/config.yaml
Prepare the development environmentWe will add a new Python based module in the IoT Edge Runtime that communicates with the Sense HAT. For this purpose we need to develop a custom module using Visual Studio Code.
1. Be sure that you have prepared your development environment before beginning this section.
- Be sure you have installed Visual Studio Code.
- Install Azure IoT Edge extension for Visual Studio Code.
- And the Python extension as well.
- Install Docker to build the module images.
- Install Python for your development machine.
- And Pip tool for installing Python extensions. It might already been installed together with Python depending on the version you have chosen.
Put the python bin folder in your path environment variable. On Windows machines it is located inC:\Users\{user}\AppData\Local\Programs\Python\Python{version}
. And the Scripts folder for python installation as well where pip is usually installed. On Windows it is underC:\Users\{user}\AppData\Local\Programs\Python\Python{version}\Scripts
. Another path you need to add to path variable is where the additional scripts will be installed like cookiecutter as mentioned below. The path on Windows machines isC:\Users\{user}\AppData\Roaming\Python\Python{version}\Scripts
.
2. Run the install command for cookiecutter
for python which is used to install IOT Edge solution templates in Visual Studio Code. This command can either be run in command shell of your development machine or VS Code integrated terminal which you can access on menu View → Terminal
.
pip install --upgrade --user cookiecutter
3. Using these development tools we will create a container image to communicate with Sense HAT from IoT Edge runtime. The runtime on Raspberry Pi will get this image from a container repository which we will create just for the purpose of this project. We will use Azure Container Registry. There are of course other repositories out there where you can check in your container images for public access like Docker Hub.
The command below will create such a repository for us named sensehatContainerRegistry
. Registry name should be unique in all Azure so you would need to pick another one.
az acr create --resource-group IoTEdgeResources --name sensehatContainerRegistry --sku Basic --admin-enabled true
3. You can retrieve the credentials for the admin user by issuing the following commands. Save these in a secure place as you will need them later on to connect to the registry.
az acr credential show --name sensehatContainerRegistry --query "username"
az acr credential show --name sensehatContainerRegistry --query "passwords[0].value"
Python based IoT Edge module for Sense HAT1. In Visual Studio Code we will create a new Azure IoT Edge Solution and a Python Module. But we need to begin by signing into Azure from Visual Studio Code.
You can issue commands inside VS Code by either the menu View → Command Palette
or by simply pressing F1
. Pick the command Azure: Sign In
and follow the instructions.
2. Then create an Azure IoT Edge Solution by issuing the command Azure IoT Edge: New IoT Edge Solution
. You will need to
- select a folder,
- give your project a name (I will use
SenseHatEdgeSolution
), - pick the Python module from the list of available module templates,
- give the module a name as well (
SenseHatPythonModule
in this example), - and provide the url for the container registry (
sensehatContainerRegistry.azurecr.io/sensehatpythonmodule
in my case)
3. Another command we need to use is Azure IoT Hub: Select IoT Hub
. This will fetch the IoT edge device we have registered on the IoT Hub and show the status of the modules running on the IoT Edge Runtime. We will even be able to get status updates, monitor the device and push new modules from inside the VS Code. After issuing the command in Command Palette
you need to select the subscription you used to create the IoT hub and then select the specific IoT Hub.
4. To check if the container registry credentials are correctly provided by VS Code open the .env
file and check that username and password fields are filled in.
CONTAINER_REGISTRY_USERNAME_sensehatcontainerregistry=sensehatContainerRegistry
CONTAINER_REGISTRY_PASSWORD_sensehatcontainerregistry=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
5. Add the following import statements at the top of the main.py
file.
from sense_hat import SenseHat
import datetimeimport json
6. Add a new counter for sense hat measurements in global counters section in the same file.
SEND_SENSEHAT_CALLBACKS = 0
7. Add the following python function after the declarations which reads Sense HAT sensors and sends a message to the output named output2
.
# read sense hat and senddef read_and_send_measurements_from_sensehat(hubManager):
global SEND_SENSEHAT_CALLBACKS
sense = SenseHat()
sense.clear()
temperature = sense.get_temperature()
temperature_h = sense.get_temperature_from_humidity()
temperature_p = sense.get_temperature_from_pressure()
humidity = sense.get_humidity()
pressure = sense.get_pressure()
timeCreated = datetime.datetime.utcnow().isoformat()
MSG_TXT = "{\"temperature\": %.2f,\"temperature_h\": %.2f,\"temperature_p\": %.2f,\"humidity\": %.2f,\"pressure\": %.2f,\"timeCreated\": \"%s\"}"
msg_txt_formatted = MSG_TXT % (temperature, temperature_h, temperature_p, humidity, pressure, timeCreated)
message = IoTHubMessage(msg_txt_formatted)
hubManager.forward_event_to_output("output2", message, 0)
SEND_SENSEHAT_CALLBACKS += 1
print ( " Total sensehat messages sent: %d" % SEND_SENSEHAT_CALLBACKS )
8. Add the following line of code in the main
function of HubManager
class inside the while
loop. Together with the sleep
statement that was already in there this will cause the Sense HAT measurements to be sent to the output2
almost every second.
read_and_send_measurements_from_sensehat(hub_manager)
9. Now open the deployment.template.json
file and locate SenseHatPythonModule
. We will add a new create option to give this module privileged rights when running in the runtime container to access the hardware of Pi. Change the createOptions
part as below.
"createOptions": "{\"HostConfig\":{\"Privileged\": true}}"
10. The default configuration for the template project creates images for amd64 architecture. But the PI has arm architecture. We need to change this default setting. In deployment.template.json
file locate the SenseHatPythonModule
and change the image setting as follows.
"image": "${MODULES.SenseHatPythonModule.arm32v7}",
11. Communication with SenseHat requires its own libraries from python code. We can include them in the container image using the Dockerfile.arm32v7
. Open that file in VS Code and add the following line of code inside the first RUN command.
apt-get install sense-hat && \
The new RUN part of the script looks like this
RUN apt-get update && \
apt-get install -y --no-install-recommends libboost-python1.62.0 python3-pip libpython3-dev && \
apt-get install sense-hat && \
rm -rf /var/lib/apt/lists/*
12. Docker creates a user to run the container as. This prevents the privileged rights to be given to the module. We can change this behavior by commenting out the following lines in Dockerfile.arm32v7
.
#RUN useradd -ms /bin/bash moduleuser
#USER moduleuser
13. Now we will build the solution and push the image to the container registry we have created. First we need to login to the container registry using the following command in VS Code Terminal. You need to replace the user name and registry url with the ones you have copied in previous sections. This command will ask for the password as well.
docker login -u sensehatContainerRegistry sensehatcontainerregistry.azurecr.io
14. In VS Code right click the deployment.template.json
file and select Build And Push IoT Edge Solution
Do not forget to Save all your files before the Build command
15. The next step is to deploy the changes on to the Raspberry Pi. Right click your device in VS Code Azure IOT Hub Devices list and select Create Deployment For Single Device
. You need to locate the deployment.json
file created in the previous step located under config
folder. Note that this file might have different names depending on the installation version.
Each time you change the contents of the project it is good practice to change the version of the container in module.json
file to better observe when the changes are deployed on the device during the development phase.
16. When the deployment finishes you can see the messages sent to the IoT Hub in VS Code by right clicking on the edge device under Azure IoT Hub Devices explorer window and selecting the Start Monitoring D2C Message
command.
[IoTHubMonitor] [6:55:31 PM] Message received from [myEdgeDevice/SenseHatPythonModule]:
{
"temperature": 43.39,
"temperature_h": 43.39,
"temperature_p": 42.34,
"humidity": 17.05,
"pressure": 1012.86,
"timeCreated": "2018-11-29T17:55:31.123579"
}
Comments