Urinalizer: Personal Healthcare Device with ML

Introduction

Some of the most widely used personal healthcare devices are Glucometer, Pulse Oximeter, Sphygmomanometer, which people use to measure blood sugar, blood O2 level, blood pressure respectively. I am trying to build another device in this category which will analyze urine with spectroscopy, called Urinalizer. With the help of Color Light Sensor and Machine Learning, this device will be able to quickly analyze urine for potential cause of abnormality. The idea is to sense urine color and apply ML to quickly diagnose/identify the underlying cause of color change.

Video Demonstration

Theory of Operation

The Nordic Thingy 53 is equipped with nFR5340 BLE SoC and bunch of onboard sensors for collecting data samples. One of these sensor is Light Color Sensor. I will be using the Color Sensor for this project.

Basically the Light Color Sensor works with the help of an RGB LED and a light pipe. When sampling for color data, the RGB LED will flash 3 times to produce RED/GRENN/BLUE light. This light will pass through the light pipe and reflect off the surface of any object placed on the Thingy 53, then the light reflected from that object will pass through the light pipe again and reach the Color Sensor inside the Thingy 53.

The sensor can extract the Red, Green and Blue components of the objects color from the reflected light.

If you observe the above gif image, you will notice something interesting ! When the Red and Green LED glows, a good portion of the Red and Green light passes through the urine sample to the top but when the Blue LED glows, most of the Blue light gets absorbed.

Any color of a substance will have an unique combination of reflection, refraction and absorption of Red/Green/Blue component.

Light Color Sensor can detect this changes and identify color.

Preparing Urine Samples Data

Since I don't have access to hospital or pathology lab, it is almost impossible for me to collect real urine samples of different colors representing different health issues.

So, I have prepared myself with some 'fake urine' samples to demonstrate the idea. These fake urines are made using Energy Drink, Soft Drink, Coffee, Floor Cleaner and Water to emulate urine samples with different issues.

From Left to Right these samples represent :

• 1. Normal or Hydrated Urine
• 2. Yellow Dehydrated Urine
• 3, Reddish Urine with Blood / UTI (Infection)
• 4. Greenish Urine with Dye from Food/Medication
• 5. Dard Brown Urine with Toxin / Liver Problem
• 6. Cloudy White Urine with Protein / Kidney Disease

Urine color chart Reference

The Thingy:53 comes with a pre-installed firmware which can access all the onboard sensors to collect sample data out-of-the-box. This makes building ML projects so easy but you can always write your own custom firmware with nRF SDK if needed. For this project, the factory firmware is adequate to do the job.

nRF Edge Impulse app

nRF Edge Impulse app is a mobile app available for both Android and iOS platform, which will work the the pre-installed firmware on Thingy:53.

This app will wirelessly connect over BLE to the Thingy:53 and it will connect to the Edge Impulse Platform over the Internet, creating an IoT bridge for sampling and uploading sensor data directly to Edge Impulse.

The first step would be to download nRF Edge Impulse app from Play Store on an Android or iPhone. Create an Edge Impulse account and Login

Next tap on "+Create New Project" and give it a relevant name.

Now, power on the nRF Thingy:53 development board.

Turn on bluetooth on phone, the device will appear inside nRF Edge Impulse app

Next, tap on connect so the Thingy:53 gets connected to Edge Impulse services through the App. This step requires an active internet connection, since sensor data will be directly uploaded to Edge Impulse.

Now, on the Data tab tap on the "+" to add sensor data

Next, select sensor as "Light", type an appropriate label that represents this data (here "Dehydrated Urine" for example), set sampling duration to 2000 ms and sampling frequency to 5 Hz.

Place the appropriate urine sample on the Thingy:53 and tap start sampling. Repeat this process 10 times for each urine sample type. Total 60 times with 60 samples (10 for each) for 6 types of urine category.

Since I have no access to real urine samples, I diluted my fake urine samples by adding little bit of water (about 15ml each time) to create more samples for collecting data.

Generating ML Model with Edge Impulse

Once all the different colored urines are sampled using the Thingy:53 and nRF Edge Impulse app, all the data will become available on the Edge Impulse web account.

In Data Acquisition section, RGB data collected using the urine samples are available

Following image shows how different urines with different issues interact with light and produce different RGB profile.

Strength of Red/Green/Blue component varies with different urine samples, this is the key to apply Machine Learning to create models that can identify urine samples once trained.

For example, according to this model :

• Healthy urine has higher Green component and medium Red/Blue component
• Dehydrated urine has high Green, medium Red and very low Blue component
• Kidney disease urine has medium Green and low Red/Blue component

Similarly, urine with other issues will have different Red, Green, Blue composition.

Next step is to Create Impulse from Impulse Design tab. RGB data from samples are provided as Time series data. As for processing block Flatten is selected and for learning block Classification (Keras) is selected.

Now. using these parameters on the Flatten section features are generated

Notice that, after generating features, most sample representing their specific label clumps together (same color dots)

Once the features generation step is completed, in the NN Classifier section the model needs to be trained with proper Neural Network settings.

• Number of Training cycles : 1500
• Learning Rate : 0.0005
• Validation Set Size :60%

Note that, these setting can be tweaked to optimize the model if needed.

Next, EON Tuner is performed to tune the ML model for target specific platform. This will optimize the ML model for platform specific mcu, making sure that the model can run properly with certain RAM, Flash and CPU limitations of that microcontroller under certain time constrain.

• Target MCU : Nordic nRF5340DK
• CPU Speed : 128 MHz
• RAM : 512 kB
• ROM : 1024 kB
• Time to Inference : 100 mS

Then, the model is retrained with all the applied changes

Finally on the Deployment tab the ML model is build while selecting the platform as Nordic Thingy:53 to generate binaries which can be flashed on the device.

Up until now, everything is done on the Edge Impulse developer web account. Next part is done from the nRF Edge Impulse app.

While Thingy:53 is connected via bluetooth to the nRF Edge Impulse app, go to Urinalizer >> Deployment and tap on Deploy to flash the ML model on Thingy:53.

This step will take some time depending on the model size.

Once all the steps of deployment is done successfully, the Thingy:53 is ready to inference urine samples.

Go to inferencing tab and test if it working properly (as shown on the demo video)

Edge Impulse Project Link

Here is the public link of Edge Impulse for this project

https://studio.edgeimpulse.com/public/135740/latest

Caveats & Conclusion

Interestingly different ambient/artificial light source can interfere with inferencing. As some light sources have certain light color components with stronger amplitude. A simple solution to this issue is to implement a consistent light source with balanced color components into the final product, plus an enclosure to block light from external sources.

This project does not use real urine to produce the ML model, it's just a concept prototype only. To make a final product one must use real urine samples, advice from Scientists, Urologist and Machine Learning Experts and recreate a real ML model for real urines.