HabitCoatchi

HabitCoatchi

After Giorgio's initial idea, Kiara decided to join him in developing this project. Pietro and Annachiara also joined, contributing through suggestions about the machine learning model and developing this description for the submission of the project.

What is HabitCoatchi all about?

The objective of this project is to create software that recognizes the habits of a user and helps them improve these habits by displaying pixel art, which serves as a visual cue to indicate whether they are adhering to or deviating from their intended habits.

HabitCoatchi's main use case

• A user works with a computer.
• If the user remains don't drink for too long, the software shows an alarm with a pixel art reminder to "drink water."
• The user responds by taking a glass (or a mug, a bottle etc.) and showing it to the webcam preparing to drink from it.
• The Neural Network running on the Ryzen AI dedicated hardware recognized the glass (or mug, bottle etc.).
• The system returns to its normal working condition, monitoring users for further drinking reminders.

The Minisforum UM 790 PRO

To ensure efficient performance and smooth operation, HabitCoatchi is designed to run on the Minisforum UM 790 PRO, a compact yet powerful mini-PC. This device provides the processing power and portability to make HabitCoatchi a seamless addition to any workspace. Its robust hardware supports the intensive tasks required for real-time habit monitoring and machine learning computations.

The Machine Learning Model

The core of HabitCoatchi's functionality relies on a machine-learning model. This model is trained to recognize patterns in user behavior using data collected via the computer's webcam. By identifying periods of prolonged inactivity, the model can trigger appropriate reminders to encourage healthier habits. The machine learning algorithms utilized include:

• ResNet-50: An existing convolutional neural network model renowned for its deep learning capabilities and pre-trained on the ImageNet dataset. Weights taken from this github.
• ImageNet: A comprehensive image database organized according to the WordNet hierarchy, which includes hundreds of thousands of images depicting various objects, particularly nouns. ImageNet has significantly contributed to advancements in computer vision and deep learning research, and its data is freely available to researchers for non-commercial use.

For HabitCoatchi, specific labels from the ImageNet database have been selected to recognize common drinking containers such as bottles, beer bottles, cups, and glasses, ensuring detection and appropriate reminders.

Instructions

To run HabitCoatchi:

• Follow the setup for the RyzenAI hardware and create a conda environment.
• Download and unpack the attached zip.
• Install or update the following libraries: PIL, cv2, and tkinter.
• Run the HabitCoatchiLauncher.py script, which will take care of launching (in multithreading) the following three scripts needed by HabitCoatchi:

HabitCoatchiStateMachine: As the name implies, this is the state machine usedfor HabitCoatchi. In line 44, it is possible to modify the behavior of the drinking state. For example, the first number in the drinking element (10800) is the timer in seconds that triggers the thirsty animation. The second number, 5, is the initial timer value (so at the first run, the thirsty animation will be shown after 5 seconds).

GraphicalProcess: this script visualizes the correct gif when instructed from HabitCoatchi's state machine.

ONNXNeuralNetwork: this script executes the neural network on the RyzenAI dedicated hardware. It then checks if the output label is one of the ImageNet labels compatible with drinking (bottles, mugs, etc.) and, in this case, communicates the answer to HabitCoatchi's state machine.

All the communications between the various processes are executed through socket transmissions.

The software is as generically as possible to allow for easy upgrades in the future.

Conclusions

HabitCoatchi aims to blend productivity enhancement with health and well-being reminders through a user-friendly interface that employs engaging pixel art. By integrating advanced machine learning techniques, the software not only monitors user habits but also encourages positive changes, making it a valuable tool for anyone who spends significant time working at a computer.

Possible follow-ups

• Expanding Habit Recognition: Incorporating posture correction, eye strain prevention, and regular stretching exercises.
• Multi-Platform Support: Developing versions of HabitCoatchi for other operating systems and mobile devices.
• Community Features: Allowing users to share custom pixel art reminders and habit improvement tips within a community.
• Gamification: Introducing a rewards system where users earn points or badges for consistently following healthy habits.
• Integration with Wearable Devices: Enhancing the accuracy of habit monitoring by integrating data from wearable fitness trackers and smartwatches.