Derma-Sense

Background

It is easy to see the beginnings of things, and harder to see the ends.

At least, according to Joan Didion. But this is exactly what I felt while working on this project.

Things got interesting sometime back when I was gripped by a question - "How do I take this ahead?" in better words - "In which direction should I take this ahead?". Yes, I had options, but none seemed *good enough* to take up. You'll get to know why.

I was already tinkering with the AMS Spectral triad long before the Advanced Wearables contest was announced, to collect the spectral signatures of different materials, for eg. soil samples, lighting, eatables like fruits and vegetables, liquids, etc.

So, when I applied for the hardware, I thought of using this sensor to collect some data either from a fluid, like blood or a surface, like skin to shape it into a sort of health monitoring system. I had done some initial research, went through some papers and articles to make sure that this wasn't just some leap in the dark.

Introduction

Spectroscopy is the study of the interaction between electromagnetic radiation and matter at various wavelengths. In this project, a spectral sensor was used to illuminate the human skin with different radiations (light) and study the interaction as a function of wavelength.

The human skin has a complex multilayered structure composed of many components and chemical substances, each having its own optical characteristics which produce different spectral responses under radiation. We can use this response, which is basically either absorption or reflection of that particular wavelength to observe changes in the skin or blood. I used three types of electromagnetic radiations - visible light (VIS), near-infrared (NIR), and ultraviolet (UV) for this purpose.

Coarsely, I started with this - first obtain some kind of information on the concentrations of biological chromophores, starting with - oxy- & deoxyhemoglobin (HbO2 and Hb) and melanin. If successful, correlate the results with other parameters or move to the next set. Then finally shape this to predict, investigate, monitor, or diagnose any underlying condition that is hidden to the human eye.

But as I went on, read more on the topic, and tried few things it became clear that this was not going to be a project wherein you build X to solve Y, it wasn't that simple. Over time, it got divided into two equally important parts - collecting the spectral samples efficiently, if possible, from multiple sources/devices. And the processing part - to select specific features from the raw data to further classify or bring in some kind of automation.

My approach

This started as a remote patient monitoring device, as I had applied for hardware bundle B, but I realized later on that it can be shaped like both smart wearable or a monitoring device, based on the kind of information it provides, so I have kept this open-ended.

The system is arranged into three parts:

• A: The wearable or monitoring device that is worn by the user. This device should be battery-powered and holds all the sensors required for monitoring etc. It will be based on the nRF5340 SoC as the main MCU with capabilities like Bluetooth, Zigbee, NFC.
• B: This is the gateway device that can connect to multiple child nodes (A). It will have multiple communication interfaces like Wi-Fi, cellular, NB-IoT, and so on, to connect with the external world over the internet. If node (A) is a smart wearable, these features can be added inside it or be replaced with a mobile app.
• C: The final place where all this data is relayed for processing. Either to a worksheet or to an application hosted somewhere on the internet where we can run algorithms to process or analyze the data. This can be bundled with A & B to create an edge device capable of the same functionality.

nRF5340DK + Spectral triad

Let's start with the two main components for this project:

• nRF5340 development kit housing the nRF5340 SoC, and
• SparkFun Triad Spectroscopy Sensor based on AMS AS7265x

nRF5340

This board is programmed using Segger Embedded Studio which comes along with the nRF Connect SDK installation. Once we are able to run some sample codes provided by Nordic, we can then make further modifications to it as required as needed. To send any string via BLE to the nRF52840 Dongle we need to make sure that the Nordic UART Service is present in the project.

To get it running either drag and load the hex file directly on nRF5340DK by plugging it into the computer. Or follow a few simple steps to build the project from the source.

- Open Toolchain Manager

- File → Open nRF Connect SDK Project → Check the configuration

- Select Projects and Board Name

- Goto Build → Build & Run

The sensor can communicate on I2C as well as UART using an AT command set. There is an Arduino library but instead of porting it I used the AT commands directly and it worked well. The pins PO.20 nRF53_TXD and PO.22 nRF53_RXD were used to communicate with the senor over UART and power supplied from a 5V pin. All this after dropping it to ~3.3V.

This is a simple setup, I have powered the MCU using a power bank and the box houses the spectral sensor and circuit for voltage step-down.

AMS AS7265x

This sensor module from Sparkfun incorporates 3 AMS chips to create an 18-channel multi-spectral sensing array covering wavelengths from 410nm to 940nm in steps of ~25nm. That is - for the fixed wavelengths of 410nm, 435nm, 460, and so on.

• The AS72651 covers NIR wavelengths from 600nm to 870nm (master controller for the set)
• AS72652 (for spectral response from 560nm to 940nm) and the
• AS72653 (for spectral response from 410nm to 535nm)

The board has 3 LEDs for illumination - White, IR & UV, which are controlled by the AS7265x device from its integrated drivers with programmable current and can be timed for electronic shutter applications.

I used the following steps for taking readings:

• Shield the sensor from ambient light using an enclosure. Keep an aperture for the sensor to take readings such that the distance between the specimen and sensor is around 1-2 inches, and cover the inside of the enclosure with black/absorbent material to reduce any interference
• Check for any bias by converting the sensor and taking a sample. Spectral intensities of all channels should be zero.
• Use any standard metal/object/calibration plate to normalize sensor readings or calibration of the sensor.
• Use combinations of LEDs to get a more vivid spectral response - for eg. UV-IR-VIS, UV-IR, IR-VIS, UV-VIS, UV, IR, VIS, at different intensities, and so on...
• Look for any spikes at some particular wavelength (where you can expect them) instead of analyzing the whole response
• Use medium gain settings (16x) and a high integration time(714ms) for better results

nRF52840 Dongle & WeMos D1 Mini & SIM800L

This setup acts like the central Gateway for the system, providing access to the internet via different communication interfaces like Wi-Fi, Cellular - 2G, and NB-IoT. Here the nRF52840 Dongle receives data from the nRF5340 DK and sends it to WeMos D1 mini over UART, just relaying everything it receives over BLE.

This setup can also be replaced by a mobile app that can listen for Bluetooth connections and send the data over the internet, I am not sure if it would work in a mesh or for multiple clients.

Now we need to connect to the internet. For this, we'll use a cheap GSM/GPRS module based on Simcom SIM800L. This is a bang for the buck *if you can make it work* with a very small footprint. The only problem with this is that it requires a very specific voltage level to work - 4 to 4.2V, and compensate the current spikes of up to 2A in transmission bursts. Else, it keeps resetting and will annoy a lot.

I tried using the NB-IoT SIM7020 but it requires a different Sim card to be bought from the telecom company. And I did not go for Wi-Fi because I wanted this to work in remote/mobile places with no Wi-Fi.

For the end-use, we need a way to transfer this data over the internet to a deployed application on a server/PaaS provider like Heroku, or else send it directly to your local machine. I choose the latter as I was not sure how I would be processing the data further. So I sent it to my laptop, by running a node.js server and exposing its IP over the public internet using ngrok. This IP address is consumed by WeMos as the target URL to send a JSON string containing all the data.

Power Profiler Kit II (PPK2)

This is a cool device to have handy if you are working on some power-conscious or battery-based, ultra-low-powered stuff. It can provide precise output voltage up to 5V and is really helpful in measuring and charting the current consumption of your DK or any other hardware.

Without this piece of hardware, I wouldn't be able to debug the reset problem with SIM800L. There is a spike in current whenever the GSM module attempted to register on the network or send any data. This spike causes a drop of voltage for an instance that is too short to register on a multimeter. PPK2 has a high sampling rate that could detect it easily. So, I used an arrangement of capacitors and diodes, to compensate for it.

This also helped me to estimate the battery life for the device by measuring and averaging the current consumed in one cycle or a time period.

Observations, Results, and Next Steps

I was only able to collect some samples of my family members and me, with some variations. The graphs are the average of all those readings with different illumination.

As you can see that even for fixed wavelengths there are peak differences that relate to certain molecules or bonds present in the skin, at various depths. This can be further studied and matched with clinical results for validation.

Next Steps

We need to consider the dynamic nature of human skin observed for each subject variability. Lack of a vivid database is the main hindrance for such kind of experiments.

Going ahead we can use Principal component analysis (PCA) to determine which parameters to be selected for prediction, use Matlab or GNU Octave for some kind of correlation analysis between samples. For comparison of two spectra, we can use the Pearson correlation coefficient. However, for a more simple method but not so reliable we can use a covariance matrix.

We can also include some pre-treatments of spectra (smoothing, normalization, centering, detrending, derivatives).

Some other things we can experiment with like:

• Illumination control using an external light source
• Effect of incandescent source on IR readings
• Make this work without heavy shielding

Conclusion

We have constructed a system capable of providing measurements of the reflectance spectrum of human skin. The initial data that we gathered shows that despite variations in subjects, skin exhibits a certain pattern for certain activities - which indirectly tells about the inner bio-chemistry. Furthermore, the pattern is directly tied to the human skin composition.

Additional analysis and more measurements are in progress. I am collecting data from different parts of the human body and also the reflectance data over a variety of age groups.

If more such sensors are commercialized, at a cheap cost, hobbyists can take advantage of the knowledge of skin reflectance.