Hardware components | ||||||
 |
| × | 1 | |||
In today's era, the merging of technology and healthcare has brought forth pioneering solutions that transform the landscape of medical treatment. The intelligent, personalized healthcare system, based on AVR-IoT Cellular Mini, epitomizes an advanced fusion between information technology and personalized healthcare. This state-of-the-art system leverages the capabilities of the AVR-IoT Cellular Mini, an embedded development platform that seamlessly integrates microcontroller functionalities with cellular connectivity. This solution establishes an unparalleled ecosystem for personalized healthcare and continuous monitoring by incorporating sophisticated sensors and cutting-edge artificial intelligence algorithms. Some of the problems addressed by this Intelligent Personal Healthcare System, utilizing AVR-IoT Cellular Mini, include:
Remote Patient Monitoring: The system can track vital signs, such as heart rate, blood pressure, and temperature, enabling remote monitoring of patients. This is especially beneficial for individuals with chronic conditions or elderly patients who require continuous supervision.
Real-Time Data Management: The AVR-IoT Cellular Mini facilitates real-time data transmission through cellular connectivity. This allows healthcare professionals to access critical patient information remotely, optimizing decision-making and providing faster responses in emergency situations.
Integration of Advanced Sensors: The solution incorporates advanced sensors to accurately measure and record medical data. This ensures detailed information about patients' health, contributing to more precise diagnosis and customization of treatment plans.
Empowerment of Patient Autonomy: Remote monitoring gives patients greater autonomy by allowing them to receive personalized healthcare from the comfort of their homes. This improves patients' quality of life and alleviates the burden on traditional healthcare systems.
Alerts and Notifications: The system can generate automatic alerts and notifications upon detecting anomalies in vital signs. This enables rapid intervention in critical situations, enhancing the effectiveness of healthcare.
Enhanced Accessibility to Healthcare: This system helps reduce distances by providing access to basic healthcare services, particularly in remote or underserved areas. Patients can receive consultations, check-ups, or guidance without physically visiting healthcare facilities, improving accessibility to healthcare.
Data-Driven Insights: The collected data can be analyzed to gain insights into individual health trends. This enables the generation of personalized recommendations and predictive analysis to prevent potential health issues. Data-driven information allows for more informed decision-making for both healthcare professionals and patients.
What will you build to solve this problem?The challenge we face lies in the lack of adequate information available to doctors during patient consultations. This deficiency is attributed to the progressive loss of essential data over time, which is crucial for accurate diagnoses. Critical information such as blood pressure, heart signals, heart sounds, or blood sugar levels is often lost. These data are vital for early disease detection and informed diagnoses.
In contrast to existing systems, the proposed solution in this scenario aims to address this situation by leveraging the AVR-IoT Cellular Mini system (Figure 1). This system is designed to collect these crucial health data and transmit them to a central server. Once received, the server uses deep learning models to analyze the data, assessing any potential health risks or issues for the user. This innovative approach ensures the preservation and analysis of vital health data, allowing for timely risk detection and improved diagnostic accuracy.
The heart of the project, the AVR-IoT Cellular Mini systemwill handle the acquisition, preprocessing, and sending of data to the cloud. The system will acquire ECG signals, among others, and send them to a server for analysis.
SIM CardTo enable our board to establish an internet connection via the cellular module, it is essential to register the SIM card. The board also supports other SIM card providers, and you can use your own SIM card if preferred. The Microchip kit includes a handy prepaid SIM card, serving as an excellent starting point.
The official setup guide outlines the following steps:
1. Confirm that you are in an area with Truphone's LTE-M coverage, which can be verified on Truphone's website.
2. Visit https://web.truphone.com/connectit/ and begin the registration process.
Upon accessing the website, you will encounter a page similar to the following:
AssistantThe assistant is composed of a series of hardware components, which when combined, enable the capture, communication, and analysis of ECG signals.
The first of these hardware components and the heart of this assistant is the AVR-IoT Mini Cellular. This development system, created by Microchip Technology (EV70N78A), is a comprehensive platform that can be used to evaluate and develop IoT solutions. It incorporates the AVR128DB48 AVR® microcontroller, the secure element ATECC608B, and the Sequans Monarch 2 GM02S cellular module. Sensors and actuators can be added using the Feather-compatible edge connector or the Qwiic® I2C connector. The development board, compatible with Amazon Web Services and Arduino, also includes a Truphone SIM card.
This development system features the following characteristics:
- AVR128DB48 AVR® microcontroller
- ATECC608B CryptoAuthentication™ device
- 25CSM04 serial EEPROM 4Mb
- GM02S cellular module
- 824-2170MHz cellular flexible antenna
- Prepaid SIM card from Truphone 150MB data included
- Nano SIM card holder
- eSIM footprint
- 5 user LEDs
- 2 mechanical buttons
- VMK3-9002 32.768kHz crystal
- VEML3328 color sensor
- MCP9808 temperature sensor
- Adafruit feather-compatible edge connector
- Qwiic® I2C connector
Onboard debugger
- Board identification in Microchip Studio and Microchip MPLAB® X
- One green power and status LED
- Virtual serial port (USB CDC)
- Two logic analyzer channels (debug GPIO)
- Onboard debuggerBoard identification in Microchip Studio and Microchip MPLAB® XOne green power and status LEDVirtual serial port (USB CDC)Two logic analyzer channels (debug GPIO)
- Compatible with the cellular Arduino library
- USB, battery, and external-input power options
- MCP73830 Li-ion/LiPo battery charger, with charge status, LED
- Two fixed power domains: 1.8V and 3.3V
In this project, the AVR-IoT serves as the heart of the system, responsible for controlling the capture of the heart signal, sending this signal to the web server, and receiving the analysis result. Additionally, it is responsible for sending this result to the next subsystem, the M5Stack Core S3.
The assistant was built using an M5CoreS3 (Figure), communicating with the AVR IoT using the I2C port. The M5 Stack CoreS3 is the third generation of the M5Stack Core series. Powered by the ESP32-S3 solution, this kit features a dual-core Xtensa LX7 processor running at 240MHz. CoreS3 is equipped with built-in Wi-Fi functionality, allowing seamless connectivity. It has 16MB of built-in flash memory and 8MB of PSRAM, providing ample space for program storage.
CoreS3 offers convenient programming options through its TYPE-C interface, supporting OTG and CDC functions. This enables easy connection to external USB devices and hassle-free firmware programming. CoreS3 features a 2.0-inch capacitive touchscreen IPS display protected by high-strength glass material. Additionally, it integrates a 30W GC0308 pixel camera at the bottom of the screen, accompanied by an LTR-553ALS-WA proximity sensor for enhanced functionality. Power management is handled by the AXP2101 power management chip, which utilizes a 4-way power flow control loop for efficient power distribution.
The assistant is designed to be extremely user-friendly, reducing resistance towards the device. To achieve this, it incorporates animated eyes that are not only decorative but also can move and blink. This feature has been strategically implemented to enhance interaction and create a more natural and pleasant connection with the user.
These animated eyes are not merely visual elements but play an active role in reflecting certain states and responses of the assistant. By moving and blinking in sync with interactions, they provide the device witha sense of life and personality, thus contributing to a more immersive and comfortable user experience.
This design choice aims to humanize interaction, allowing the user to feel more comfortable interacting with the assistant. Incorporating this visual dimension aims to create a deeper and more positive emotional connection, reducing potential apprehension and fostering a friendly relationship between the user and the device.
When the user touches the assistant's screen, it displays a menu. The user can find a series of options related to different measurement devices in this menu.
The Figure illustrates the menu available to the user for interacting with the assistant. This menu serves as a central hub through which users can navigate various functionalities and commands offered by the assistant. Its intuitive design facilitates seamless interaction, allowing users to access and utilize the assistant's features easily.
Users can initiate a wide range of actions from this menu, and each option is carefully organized and labelled to provide clarity and ease of use.
Moreover, the menu is designed to adapt to the user's preferences and needs, offering customizable features such as personalized shortcuts, theme options, and language settings. This flexibility ensures users can tailor their interaction with the assistant to suit their preferences and workflow.
Furthermore, the menu serves as a platform for the assistant to showcase its capabilities and versatility. Users can explore different functionalities and discover new ways to leverage the assistant's assistance in their daily tasks and activities.
The menu consists of 5 buttons, with the first one used to initiate the ECG signal capture process.
The ECG button starts the ECG signal capture process when the user clicks this button. The assistant has two electrodes connected to the BMD-101 to capture the ECG signal(Figure). These electrodes are strategically placed on the user's body to ensure accurate signal acquisition. The BMD-101 serves as the interface between the electrodes and the assistant's system, facilitating the capture and processing of the ECG signal.
Once the fingers are correctly positioned on the electrodes and connected to the BMD-101, the assistant initiates the signal capture process. The BMD-101 effectively detects the heart's electrical impulses and promptly converts them into digital data. Subsequently, this digital data is seamlessly transmitted to the assistant's system for comprehensive analysis and interpretation.
This device communicates via serial port with the AVR-IoT.
To perform this capture, the M5 sends a code to the AVR-IoT via I2C, instructing it to begin acquiring the signal. Upon receiving this code, the AVR initiates the data capture process, sending the acquired data to the web server. The number of data points corresponding to the ECG signal sent by the AVR is 187. These data points are then stored on the web server in JSON format, along with the associated timestamp, which allows for later identification of when the signal was captured.
The Heart Rate (HR) button, which is used to access the MAX30105 sensor. This sensor calculates the heart rate and sends the results to the AVR to be added to the HR field. Subsequently, the system sends this result to the web server for storage.
The Temperature button enables the capture of the user's body temperature, utilizing the MLX90640 sensor. This sensor enables non-invasive temperature measurement, which helps reduce potential stress that some individuals may experience with traditional methods.
The result of this measurement is transmitted via I2C to the AVR-IoT, which stores it for later transmission to the web server. This seamless process ensures efficient and accurate temperature monitoring while prioritizing user comfort and well-being.
The Diagnostics button, once the various data captures have been completed and sent to the web service, allows communication with the server to retrieve the user's diagnosis. The diagnosis is subsequently displayed on the M5 screen, indicating whether the ECG signal is categorized as Normal or Abnormal. This feature provides users with immediate feedback on the status of their ECG signal, facilitating prompt decision-making and enabling timely intervention if necessary.
Web ServerThis project features a web server built with Flask; Flask is a Python web framework that has gained popularity for its simplicity, flexibility, and ease of use. Although it labels itself a "micro" framework, this should not be underestimated, as it provides essential tools for building web applications efficiently.
Here are some key features of Flask:1. Simplicity: Flask follows the principle of "keep it simple." Its minimalist design gives developers greater control over the libraries and tools they want to use, avoiding the inclusion of unnecessary components.
2. Flexibility: Flask does not impose a rigid structure for your applications. You can choose your own libraries and tools to work with, adapting it to the specific needs of your project.
3. Extensibility: Although Flask is lightweight, it can be easily extended with various extensions. These extensions range from database integration to user authentication, providing additional modules for specific functionalities.
4. Template Rendering: Flask uses Jinja2 as its template engine, making it easy to create dynamic web pages. This separates the presentation logic from Python code, improving code maintainability.
5. Rapid Development: Flask facilitates rapid prototyping and building of small to medium-sized web applications. It is ideal for projects where the complexity of other frameworks might be excessive.
How to Install Flask:Installing Flask is quite simple using pip, the Python package manager.
pip install FlaskThe web server plays a crucial role in this project as it is responsible for receiving the data sent by the assistant via AVR-IoT. A JSON format is used for organizing the data to be sent. The structure is as follows:
[
{
"sensor_id": "AVR-IoT",
"bio_signal": [
47,
57,
63,
58,
37,
14,
10,
-262,
-258,
-243,
-216,
-180,
-131,
-69,
.
.
.
],
"pre_diagnostic": "Diagnostic:Ok",
"HR": [90],
"QOS": [1],
"temperature": [18.75,19.1],
"accelerometer": [2.76, -3.75, 0.84],
"gyroscope": [-3.78, 0.03, 0.0],
"magentometer": [475.97, 0.0, 0.0],
"timestamp": "2024-02-22T22:04:17.316841"
}
],This JSON contains data from a sensor reading. Here's a description of each key:
sensor_id: Identifies the sensor used, in this case "AVR-IoT".
bio_signal: Represents a biosignal, possibly physiological data such as heart rate or ECG readings.
pre_diagnostic: Indicates the result of a preliminary diagnostic, here labelled "Diagnostic: OK".
HR: This stands for heart rate and contains heart rate data.
QOS: Quality of Service indicator, likely related to the sensor's performance or data reliability.
Temperature: Provides temperature readings with values in Celsius.
Accelerometer: Records acceleration data in three dimensions (x, y, z).
Gyroscope: Measures orientation or angular velocity around three axes (x, y, z).
Magnetometer: Captures magnetic field data, typically in three dimensions.
Timestamp: Indicates the date and time of the sensor reading.
Overall, this JSON captures various sensor data and metadata such as sensor ID and timestamp.
The assistant sends 187 ECG signal data points to classify cardiac pathologies via JSON format. This signal is utilized by the model hosted within the server, which analyzes it and provides the user with a pre-diagnosis of their heart condition. The dataset was used to train this model at the following website.
Two models have been trained for this purpose: the first model contains two classes, distinguishing between normal and abnormal signals, while the second model contains five classes:
If the first model determines the signal is abnormal, it automatically redirects it to the second model. This second model contains a Res-Net network with more neurons and parameters. This setup allows the system to determine if the signal is indeed abnormal. Otherwise, it returns one of the four classes contained in the model.
The following Figure depicts the normalized confusion matrix obtained from the validation of the first model. It is evident that the model accurately distinguishes between normal and abnormal signals sent by the user. This performance indicates a robust learning capability and effective classification accuracy of the model.
On the other hand, the following confusion matrix shows the result of the second model, with satisfactory results in classification. Al igual que en el primer modelo, se pude observar que la diagonal de la matriz de confucion muestra valores sobre el 70%. Con esto se puede concluir que el modelo clasifica de forma satisfactoria las clases N, V y Q. No obstante las clase S y F, presentan un porcentaje de clasificacion sobre el 70%. Esto se deberse a la existancia de un desbalanceo de los datos para estas señales o que es necesario realizar un pre-procesado de señal anterio al entrenamiento.On the contrary, the subsequent confusion matrix illustrates the outcomes of the second model, showcasing commendable classification results. Similar to the first model, the diagonal of the confusion matrix exhibits values above 70%. From this, it can be inferred that the model effectively classifies the N, V, and Q classes. However, classes S and F demonstrate a classification percentage above 70%. This disparity might stem from either data imbalance issues for these signals or the necessity of pre-processing signal data before training.
If the first model indicates the signal is abnormal, the system transfers the signal to the second model. This model analyzes the signal in more detail to determine if the captured signal is indeed normal or abnormal. Once the classifier has completed the data analysis, the result is sent to the assistant. This transmission occurs when the user requests the system to display the diagnosis.
ConclusionandFuture Work
This project utilized the AVR-IoT Cellular Mini system to design and develop an assistant intended for use by elderly individuals living alone or in rural areas.
The system incorporates a range of technologies to facilitate data acquisition, processing, and transmission to computing centers for analysis. Simultaneously, the assistant features a user-friendly and intuitive interface for interacting with it, aimed at reducing potential resistance to these technologies.
The prototype presented in this project can be easily compacted, allowing for the integration of all devices into a more streamlined structure. As a future endeavor, efforts are being made to embed trained models within the device itself. This way, only the signal would be sent to the server if the initial analysis detects an abnormal ECG signal.
The following figure provides a visual representation of the data traffic utilized within the scope of this project, facilitated by the Truphone SIM card. It offers insights into the flow and utilization patterns of data, showcasing the network activity associated with the project's operations. Through this visualization, stakeholders gain a clearer understanding of how data is transmitted, received, and processed via Truphone SIM, aiding in the analysis and optimization of data usage strategies.
Comments