Swimmer Assistive System

Introduction
Swimming can be challenging for people with visual impairments. They may have difficulty orienting themselves in the water and staying in their lane, without veer off course toward the lane line. They may also have difficulty knowing when they have reached the end of the pool.
We propose a Swimmer Assistive System that consists of computer vision equipment and a Bluetooth-enabled bone conduction earphone.
The computer vision equipment includes a Coral Micro Board and an ESP32 Dev Kit: the Coral Micro Board identifies the swimmer’s location, while the ESP32 Dev Kit that broadcasts audible feedback via Bluetooth.
The swimmer wears a bone conduction earphone, that is enabled with Bluetooth, to hear their location that is inferred by the computer vision equipment.

Swimmer Assistive System
There are two pieces of hardware in the swimmer assistance system. The first piece of hardware is a computer vision device that monitors the swimmer and broadcasts audible feedback. The second piece of hardware is a Bluetooth-enabled bone-conduction earphone, which is available on common e-commerce sites.

note 1: The computer vision equipment is installed somewhere above the swimming pool.
note 2: The bone conduction earphone is worn by the swimmer. A swimming cap is recommended to prevent loss of the earphone during swimming.

Computer Vision Equipment
There are two core components in the Computer Vision Equipment. The first one is a Coral Dev Board Micro, which monitors the swimmer and whose firmware is available on freertos-coral-swimmer. The second one is an ESP32 DevKit, which broadcasts audible feedback and whose firmware is available on esp32-a2dp-source.

Demo of this proof-of-concept
Since there is a camera on the Coral Dev Board Micro, it requires permission from the swimming pool administrator to use it in the swimming pool.
Instead of performing a physical test in a swimming pool, we modified the firmware on the Coral Dev Board Micro and leveraged other components to demonstrate this proof-of-concept. For example:
1. A blue box represents a lane in the swimming pool
2. A doll to represents the swimmer
3. A Bluetooth speaker to represents a Bluetooth enabled bone-conduction earphone (in order to record the audible feedback in the demo video)

Steps
1. Power up both the Coral Dev Board Micro and ESP32 DevKit
2. Launch a serial terminal and connect it to the virtual port of the Coral Dev Board Micro. If everything goes well, you should see the following on the serial terminal:

3. Launch a Ubuntu/Linux terminal and type "cd ~/freertos-coral-swimmer" followed by "python3 py/coral_swimmer.py".
4. Place a blue-colored box under the camera. If the box is detected, you should see the box inside a white frame on the GUI app. The "Status LED" on the Coral Dev Board Micro should also be turned on.
5. Power up the Bluetooth speaker and wait a couple of seconds. Once the ESP32 DevKit is connected to the Bluetooth speaker, the "User LED" on the Coral Dev Board Micro should be turned on.
6. Place a doll on the blue-colored box.
When the swimmer is in the middle, you should see a green-colored rectangle in the GUI app around the swimmer and hear a "beep" tone on the Bluetooth speaker.
When the swimmer is near the left lane rope, you should see a red-colored rectangle in the GUI app around the swimmer and hear a "bright" tone on the Bluetooth speaker.
When the swimmer is near the right lane rope, you should see a red-colored rectangle in the GUI app around the swimmer and hear a "bell" tone on the Bluetooth speaker.
If everything goes well, you should see the camera image in a Ubuntu GUI app.

System image

Video

Thank you for your time on reading this proof-of-concept.

Source code
Please refer to https://github.com/teamprof/github-freertos-coral-swimmer about the source code of firmware on the Coral Dev Board Micro.
Please refer to https://github.com/teamprof/github-esp32-a2dp-source about the source code of firmware on the ESP32 DevKit.