Haptic Vision Assist

This is my first submission to Hackster.io. My project does not fit perfectly into the swimming solution challenge, it is more of a general mobility tool for individuals who have visual impairments. My fiance and her son have a genetic eye condition called Retinitis Pigmentosa. Living with a visually impaired partner has allowed me to witness the challenges that people with vision disabilities face on a daily basis. I have worked with her closely to develop a tool that can assist her and others with visual impairments in navigating environments where obstacle avoidance is necessary. This project began before I learned about the Build2Gether challenge. Over the course of a year I have learned to use M5stack and UIflow. To learn the platform, I challenged myself to rapidly prototype an idea a day. During this time I began developing the basic framework for this project with input from my partner. My hope is that this will stand as an example of what can be achieved through a willingness to learn, diligence, and care. M5stack provides a powerful platform for rapidly prototyping and testing these kinds of devices. This project would not have been possible without the ingenuity of M5stack's products, code logic advice from Jilles Groenendijk, and the priceless input from my wonderful partner Amanda. Getting to speak with Marc Chiang was an absolute treat. He and my partner have the same eye condition. Mr. Chiang shared resources with her as well as providing input and perspective for this project. This is a labor of love. No neural networks, apart from human, were used to generate this code or design.

Summary

The haptic vision assist wearable consists of three M5stickCplus esp32 modules.

These StickC modules gather data from infrared time of flight laser sensors.

These sensors are placed on clothing via magnets and adjustable Lego mounts.

The M5StickC modules translate distance data into haptic feedback via vibration hats. (vibration motor modules)

The face obstacle detection module and sensor is worn on the chest is angled upwards in order to detect any obstacles that would interfere with the wearer's head or face.

The stick module with vibration hat can be either worn inside or outside of the user's clothing. The two ground obstacle detection modules are worn below the face/head obstacle detection module. They are pointed angled towards the ground in a cross over configuration so that each module detects a line of sight in a cone from the user's center.

The stick modules and the vibration units can be placed in the wearer's waistband on each hip, or worn magnetically inside or outside of clothing. Once the modules are in place, a baseline ground distance calibration can be triggered by a long button push of button A (labeled m5) on each module.

The stick module sounds two beeps, the first lower then a higher frequency when this calibration is set. When the calibration is cleared by double pressing the A button, the tones are played in reverse.

This sets the baseline distance to the ground so that if the distance increases by more than 2 inches, the lower body vibration motors return a rapid on off vibration pattern alerting the user that there are steps or a steep decline ahead.

To return a battery percentage on each of the three modules, the B button is pressed and the vibration motor on any of the modules returns a number of quick pulses in tenths to denote battery percentage.

If the button is pressed and the battery percentage is under 10 percent, the module will not vibrate but will beep.

Face/head module and sensor

1. Vibrates with full duty if an object is within 3-4 feet of a user's face or head area

2. The sensor must be angled properly so that at 3-4 feet, the vibration alarm initiates if an obstacle is at face/head level

Abdomen/Lower modules and sensors

1. Vibrates at different intensities when an obstacle is an front of the wearer on either hip or side of the body. This allows the wearer to determine which side of their body the obstacle is on.

2. Once the sensors are adjusted properly on the body, a long press of the A button on both/either sensor sets the baseline distance to the ground. This allows the wearer to detect decline/steps with a series of repetitive pulses

3. If the button A is pressed, the vibration motor returns battery percentages via vibration pulses in tenths

All Modules

1. Powered on by pressing the side button on the lower left ohand button of the module. Power on is followed by two fast vibration pulses.

2. Powered off by long pressing the side button on the lower left hand button of the module.3. The M5stickC module is easily connected to its respective time of flight sensor via a grove cable on port A

4. Easy charger attachment via magnetic charging cable

Lego Mounts

Uses assorted Lego Technics parts and an adhesive backed magnet. Fully adjustable and customizable to the user's needs.

1. Face/Head sensor mount

2. Lower body abdomen sensors and lanyard for support

Future Ideas/Improvements

This submission is a prototype. It uses time of flight sensors to detect objects and obstacles and translates that data into varying intensities of vibration patterns that create a sensory experience for the wearer. The hope is to continue to advance this project. I plan to add additional useful features like object recognition, facial recognition, optical character recognition, GPS navigation, and upgraded adjustable mounts for stylish wearability. I have begun development on code that utilizes an M5stack UnitV2 ML camera to count nearby faces and translate that data to haptic feedback for the wearer. This and other features currently under development have not been implemented in this iteration of the project.