A Smart Water Controller Using TinyML and Nordic Thingy:53

Saving the planet can be done in everyday life without any major efforts. For example, if you recycle your paper products, you reduce environmental impact. All of us here can not control the negative impact on environment and that’s why we should take actions to reduce those impacts.

Water is an essential resource for life, but how much does your family really need? This Proof of concept project will help you conserve water at home. Avoid wastage of water by making your home smarter. This will not only save you money. but also reduce strain on the environment.

And hence computer vision comes into play here and can help us to save water consumption in our everyday life. Here, I will use the machine learning algorithm with the help of Edge Impulseweb platform in order to create a water tap consumption model and deploy it to Nordic Thingy:53 development board.

Here is a sneak peek of what's about to happen:

• Collect dataset to Edge Impulseto train the model using nRF Edge Impulse mobile application
• Feature Extraction
• Training of the model
• Export the model to be analyzed by netron.app
• Deploy your model to Nordic Thingy:53 development board
• Real-time data evaluation and testing using nRF Edge Impulse mobile application

Nordic Semiconductorjointly withHackster.iowas kind of enough to send me a Nordic Thingy:53development board to participate in the Smarter Sustainable World Challenge with Nordic Semiconductor competition.

Hardware Requirements

For this project, I used the following hardware:

• Nordic Thingy:53 development board. It is a compact, power-optimized, multi-sensor development kit.

Nordic Thingy:53 development board has a wide range sensors, it allows you to start complex projects relatively easily, accelerate development speed, and improve quality at the same time.

It can be configured over-the-air using Bluetooth Low Energy(BLE). Unlike classic Bluetooth, which is always connected, the BLE is usually in standby (idle) mode and wakes up if and only if required. Hence, the ultra-low power consumption. With BLE, application data is sent and received using the GATT system. GATT uses services, characteristics, and attributes to organise data and control how the data can be read from and written to. This service simulates a basic UART connection over two lines, TXD and RXD.

Without further ado, let’s get started!

Create and Train the Model in Edge Impulse

Edge Impulse is a web-based tool that helps us quickly and easily create AI models that can then be used in all kinds of projects. We can create Machine Learning models in a few simple steps and users can build custom image classifiers with nothing more than a web browser. In a nutshell, Edge Impulse is a great platform for non-coders to develop machine learning models.

Go to Edge Impulse platform, enter your credentials at Login (or create an account), and start a new project.

Then, download nRF Edge Impulse mobile application to in order to collect a dataset. You can download the nRF Edge Impulse for your smartphone either from the Google Play Store or Apple App Sore.

And collect the data. We will be adding the dataset to the respective classes.

The classes are:

• High Consumption
• Moderate Consumption
• Low Consumption
• No Consumption

In my case, I uploaded 30 audio samples for each class. We should collect enough samples for each of our classes. So, in total, around 120 audio samples were taken to train the model.

It would look something like this:

Once you have set up all of your classes and are happy with your datasets, it is time to train the model. Navigate to Impulse Design on the left navigation menu.

Select Add a processing block and add Spectrogram, since it is very suitable for non human voice applications. It will try to convert audio into some kind of features based on time and frequency characteristics that will help us to do classification. Then select Add learning block and add Classification (Keras).

Once the impulse pipeline is complete, click Save Impulse. Then navigate to Spectogram. Under the Spectogram we will keep the default parameters. Click Save parameters. Then, Click Generate features to start the process.

So, it’s time to start training a neural network. Set the Number of training cycles to 200 and the Learning rate to 0.005. Next, click on Start training to train the ML model, this might take some time to complete depending on the size of your dataset.

While training is in progress, you'll see a progress bar. When the training process completes you should have a result like the picture shown below:

We get a validation accuracy of 100%. A 100% accuracy is possible, yes, but that does not mean your model is 100% accurate. If you have 100% train accuracy, you've probably massively overfit. Overfitting is a condition where a model doesn't perform well on unseen data. In other words, The AI/ML model memorizes the training data rather than learning from it. Since, it is just a proof of concept project, it can be neglected.

The .tflite file is our model. The final Quantized model file(int8) is around 36KB in size and it achieved an accuracy almost 98%. This quantized model is expected to take around 12ms in inference time, using 67.5Kb in RAM and 90Kb in ROM.

You can use a program like Netronto view the neural network. Then go to either Live classification or model testing in the left menu in order to test it.

Once the training process is complete, we can deploy the trained model to Nordic Thingy:53 development board. Navigate to Deployment on the left menu. In the Build firmware section, click on Nordic Thingy:53 Firmware and then, on the bottom, click on the Build button.

Before converting the model, you need to enable the Edge Optimized Neural EON compiler, which allows running with 25-55% less memory and up to 35% less flash compared to TensorFlow Lite for microcontrollers neural network while maintaining the same accuracy. It is enabled by default. A zip file will be automatically downloaded to your computer.

Follow instructions here to flash Nordic Thingy:53 development board. It explains all necessary steps to flash it. I recommend to use nRF Connect Programmer in order to flash the new firmware.

Select write button. While writing is in progress, you'll see a progress bar.

Finally, open the terminal, run real-time inference using below command.

edge-impulse-run-impulse

Open the serial monitor, and see some inferences. You should see some timing results along with the output predictions

Inferencing settings:
Interval: 0.0625ms.
Frame size: 16000
Sample length: 1000.00 ms.
No. of classes: 4
Starting inferencing, press 'b' to break
Starting inferencing in 2 seconds...
Recording
Predictions (DSP: 58 ms., Classification: 176 ms., Anomaly: 0 ms.):
High consumption:  0.000000
Low consumption:  0.996094
Moderate consumption:  0.000000
No consumption:  0.000000
Starting inferencing in 2 seconds...

Open the nRF Edge Impulse application and navigate your self to Inferencing section and finally click Start button.

Finally, the test video is as follows.

I feel the results of it are fantastic despite a few misclassifications.

So in conclusion, you can potentially reduce the water consumption at your home with this smart device, which is based on the Nordic Thingy:53 device. It has a simple interface that lets you set water limits for each tap. Once you turn it on, it will take into account the usage patterns of everyone using your taps and you can make adjustments accordingly.

Comment if you have any thoughts of improving it or any doubts about the process. You can follow these steps to build more advanced projects.

References

• Introduction to the Thingy:53
• Snoring Detection Using Nordic Thingy:53