--------------
I started thinking to this project moving from a question: is there a way for a visual impaired person, or with a severe sight reduction to avoid or at least be warned for a danger coming from an object moving toward him. Same risk could be a relative motion, if for example moving by walk toward a still object (a wall, a parked car, a tree, etc...).
This relative motion and can be managed the same as a moving object, the person is experimenting the relative movement, if he is still then the object is moving, if the person is moving by walk (or in other way like a wheelchair) the object could be still (as a wall or an obstacle on the walk path).
Generally visual impaired people to detect moving objects rely on the sense of hearing on perceiving ambient sounds and stimulus and this skill is greatly exercised in them, This skill works well to identify moving objects but works less or not at all if the object is still or moving silently, for example an animal as a cat (I experimented this in my test with my cat).
A major problem arises if the visual impaired person has a diminuished hearing sense, didn't wear the hearing aid device or suffering by total deaf.
I thought on what Braille code is, how touch sensation or any other vibration sensation can give as information trought the important sense of touch that is an important resource for visual impairement people. As well established touch sense helps in reading documentation or learning more about objects by touching them with the tip of fingers.
How can touch sense work in collecting information from the ambient, moving around in home or just walking on a street. Can I design an aid in detecting objects quite far away and I can't touch?.
The lack of information also applies if sitting in a park (or other outdoor activity) and something is getting near. Only hear sense could help, but is not enough if there is also a severe hearing reduction as formerly explained or there are many confusing sounds around, as children playing or car travelling on the street.
My idea was to build something that could detect and warn upon moving objects or obstacles on the path while moving toward them (considering a relative motion).
This detection could be transmitted to the visual impaired person with a vibration that is clearly sensed by the body and can perhaps be read out trought fingers. Could be just a qualitative information to warn the person about an incoming hazard or to inform that something is moving (remember it could be extremely silent object) and this could be enough for this first design of the device.
To increase effectiveness of the device I add a sound buzzer that wakes up and warns promptly the person on movement detection and then follows a shake that can be sensed trough touch sense. To complete all I added power LED flashing to warn an external object approaching (could be a second person, ora a driver on a car...) that there is a visual impaired person, be careful, stop or change direction.
I thought how could be realized this device and what type of sensor or technology could be best for the detection in any time of the day. Is there daylight or full darkness, too much light or background noise on image (trough a camera for example) or many objects moving at the same time.
This initial design phase brought me to put attention on an image camera and my initial idea of Object detection was trough video frames examination looking for moving bounding boxes. This first solution was replaced by a second solution based on microwave radar technology.
To keep open many ways to detect movement in my device I provided the basic hardware for image camera, so to work only once on project hardware letting open the device for further developments only working on coding computer vision software. All must work in the most simple and effective way and this is the direction that inspired this device and the component selection I made.
The reason I modified my first choice is that after many considerations my conclusion was that could not be the best solution due to ambient light dependancy.
Another consideration was on the hardware to process in short time the image contents and run the underlying software code on the hardware choosed for my project. My target was to use a less expensive hardware to make this device work and cost very low price and could be build in easy way, something accessible to a large platea of people. I also planned to use a small factor board as the XIAO ESP32S3 to run code of the device.
The idea was to wear the sensor on something like a cap on the head for moving detection exploration in the direction of the head. Finally I came to the final solution in making the sensor integrated in the vibrating touch pad structure all in a robust and easy to handle device. A cap could be removed from the head by a strong wind or something moving too much near. So I finally designed to wear the device in front, on the body chest just as a small jogging case for smartphone with a strap around the neck.
This makes the device more easy to be carried and points in the direction were the body is going directly just in front. This device can be also put aside the person on a table, chair or on a bench if the person is sitting in a park.
---------------------------------------------------------------------------------------------------------
HOW DOES IT WORK, PRONS and CONS for selecting the sensor:---------------------------------------------------------------------------------------------------------
After Prons and Cons evaluation I selected a Radar microwave sensor for detecting movement for the following Prons:
====================================================
1) it is not influenced by quality of image or shades;
2) doesn't depend on light intensity to work properly;
3) simple to use and develop code for it;
4) it is costless;
5) detects directly the presence of movement;
6) it is programmable to get the best performance from it;
7) acceptable time latency on detections, this time should be
made shorter as possible working with the manufacturer technical
support (DFRobot);
8) can detect up to 11 meters of distance, quite enough
for increasing safety in outdoor activity but applicable also for indoor.
===========================================================
There are also cons with this choice, but of course this is engineering, working to make always better performance over the weak points and this idea in the development step ( the first revision ) is planned to follow upgrades.
Cons are:---------------------------------------------------------------------------------------------------
1) Sometimes latency seems to be around 2/3 seconds, some work is to be done on the settings and program firmware of this board (SEN0521);
2) It doesn't have direct access to distance information but only on movement detection. For this reason I have worked on my device software to make a distance algorithm, enough for getting qualitative information.
3) It works on a front Radar detection area with a 120° angle. Further improvements could use 2 of this sensors with a small angle shift in order to work on a > 200 ° sector or greater. This upgrade could give almost a 360° warning that could be useful also for a general hazard warning and direction detection capability.
===============================================================
Let's go deeper now in the description of this sensor and working principle.
The "artificial Eye" of the device is the Microwave Radar sensor SEN0521 and it works using a 5.8 GHz microwave to illuminate the objects in front of it and get back from it a reflected radio wave. The difference between the two waves (transmitted / received) is measured over time and when the difference is meaningful for a movement then a trigger for detection is raised.
This sensor has also a serial communication port useful for setting the working parameters and for reading the detection information, alsouseful for debugging code as I reported in my papers for this work (look to instruction Manual).
The depth range is about 11 meters maximum for the sensor. I selected this distance as a tradeoff for awarness and false alarm avoidance, but of course distance can be higher if necessary. I think this distance is enough for not getting confused on a moving event too much far away, just the distance you need for safety, don't get confused on what is happening too far away.
This sensor works also if the object is almost still with a slow position variation just as a breathing body that slowly move the chest.
Once the detection is triggered the code in the control card (XIAO ESP32S3) board sends out a pulse to the vibration motor (as used in gaming controls) and the lenght of the pulse is modulated to be longer more closer to us is the moving object.
The scan distance can be varied trough the touchpad using the "distance button" pressed longer to move to higher scan distance or pressing twice in short time to make the distance scan enter a self ranging mode (it varies following a determined algorithm).
Effort was put in thinking how to build the touchpad in a simple, effective and costless manner.
After some trials the actual structure emerged composed from a main box with room for the sensor, the electronic card, battery and finally the vibrating structure made by fingers in 3D printed shapes.
-------------------------------------------------------------
3D PARTS TO BE ASSEMBLED:.------------------------------------------------------------
The sight movement awarness pad is made from several designed 3d printed parts:
.1) HACKPAD_1.OBJ: this part is the main support with vanes to insert sensor, main controlboard, XIAO ESP32S3, battery, wiring, vibration motors and buzzer.
2)HACKPAD-sotto.OBJ: back cover box for electronic circuits and battery;
3)HACKPAD_sopra.OBJ: centering support for electronic motors;
4) HACKPAD-vibra.OBJ: finger shaped vibration pad for sensing vibration intensity.
5) HACKPAD-sens.OBJ: sensor FRONT protection cover.
6) HACKPAD-lens.OBJ: protection enclosure for camera and microwave sensor.
7) HACKPAD-led.OBJ : protection for leds.
All these parts mounted achieves a small pad of 130 x 80 mm to be placed in front of chest trough a strap as in picture below showing how ro wear the sight movement awarness pad in easy way.
This device can also be fixed to a wheelchair or put near a person sitting on a bench outdoor in the park or at home indoor and doesn't require to be strictly weared, so it can be used in many ways.
----------------------------------------------------------------
HOW TO USE THE PAD:1- Once started (switch turned ON on the back of pad), the pad vibrates, sounds the buzzer and flashes the blue led (on the front) to warn all is properly working and ready to run the code.
2- The pad initialize with the maximum distance detection resolution, about 11 meters. This is the average meaningful distance for awarness without getting continuous alarms for events too far away and that are not an actual hazard to safety.
3- On any alert for moving a object detected the scan distance is varied in order to finalize if the object is getting closer or is going far away (departing).
4- The Alert animates on the pad with vibration of the fingers tuned to the alert distance, and in this way it is possible an estimate of the alert distance and the readout of it touching the finger that is vibrating.
5- This way to work is automated and the software is modifing the distance at every alert. The other way is to use the manual mode, just select trough the pushbutton, pushed twice in short time, so to vary and keep fixed the scanning distance.
6- In manual mode the distance is selected pushing one time the selection button. The selected distance is shown trough vibration of the pad. For example, push 2 times then distance is 9 meters, the pad vibrates for 9 times shortly.
7- To get back to automatic mode just press twice again the pushbutton.
-------------------------------------------------
FRONT LED INDICATORS------------------------------------------------
On the front there are 2 LED that show the status of the pad, if there is a detection they lit on in flashing squence. If no detection is sensed they are OFF.
This permits to visualize the detection state of the pad being at some distance from it and to highlight the position of the visual impaired person wearing the device to an incoming hazard. This is most valuable in a dark ambient or at night when there is scarse illumination of a street (for example).
Note that with this function the LED can emit more powerful light (if a high emission led is choosed) so to warn approaching objects (as a car or pedestrian) to be careful for the presence of a visual impaired person. This can be very useful to prevent collision hazard at dark in home or at night outside for example on a street.
-------------------------------------------------------------
FINAL IMPROVEMENTS ON PROJECT-------------------------------------------------------------
This idea was realized using the generous contribution of the project sponsors below listed. The sponsor hardware parts used for the project are:
1- Seeed studio XIAO ESP32S3 SENSE2- DFRobot SEN0521 microwave sensor3- PCBWay pcb boards servicesThe hardware and 3D construction of the device is capable to demonstrate a working device as described formerly in the project idea story to help visual impaired people to avoid collision hazard with moving objects or just to inform them that something is moving near (for example a silent animal or a person).
As I expected any time a new idea is brought to reality some enhancements to overcome some limits and better foreseen functions to be implemented are coming out. Suggestions are wellcome to make a better product to help the community and I warmly encourage them because this starting point should get better and better and go out in the community to help visual impaired people.
At this stage of first revise I identified some improvements to bring on a next upgrade:
1) faster response, less latency time between consecutive detections;
For this I must get support from the sensor manufacturer (DFRobot)
that yet didn't satisfy my request of information. This Microwave
sensor is extremely interesting and can be enhanced a lot more.
2)greater distance for detection (15/ 20 m) and shorter time to change distance setting by command issue. Extending the distance could be a request from people using my device modifying my initial project 11 meter maximum detection distance as the best tradeoff.
3)adding the code forImage camera to perform mixed detection with OpenCV open source computer vision framework for object tracking. Actually I have already placed all necessary hardware to apply this upgrade, and the coding of it has started but yet not completed in order to complete my tests.
4) adding object detection to speech translation so that information becomes of higher quality with audio output. This makes a step forward than using a simple buzzer to sound beeps on detections.
The XIAO ESP32S3 device was performing very well for this application and has sufficient I/O pins to make all working but an I/O expansion card could be useful if more fingers are to be added or other functions as flashing light alarms trough external trigger, brake driver for stopping wheelchair, speech translation of detections, signals sent trough Blues Note card in wireless mode to notify detections to people far away in order to assist visual impaired people.
As described, initially the project was thought to run the OpenCV image processing library but this choice was not applied for the following reasons (but still keeps in my interest to upgrade the project):
1- a more powerful and fast processing unit was necessary.
2- time to get the correct information could be greater than the one demonstrated by the microwave sensor.
3- light conditions as darkness, light flares, out of image focus could modify the detection process. Could be used a IR image sensor but could exist same limitations as the listed.
By installing and running succesfully the OpenCV video-image processing could permit to do a image Classification of the object as:
a- car/vehicle
b- person/ pedestrian
c- animal (if not a person --> probably is an animal)
d- wall/ physical barrier.
Maybe I should use a more performing board to do this final upgrade and some more time and effort is necessary to get this. This is a challenge for my idea and I encourage for proposals in this direction.
Final device designing my own 3D parts and internal hardware circuit
Comments