Build2gether2.0 – MOBILITY IMPAIRMENTS Track1: Entry Project

Introduction

This document details a project proposal for the Build2gether 2.0 challenge focused on assisting individuals with mobility impairments. The project aims to create a device that guides wheelchair users in unfamiliar accommodations to their desired location, avoiding obstacles and tight spaces. A hotel Smart Wheelchair Navigation system that uses sensors and GPS to map out a safe and efficient route for wheelchair users and an Obstacle detection system with Sensors installed in hotel rooms to detect obstacles and alert users, helping them navigate safely.

The device has two modes of operation: Learn Mode, where it maps the environment, and Run Mode, where it provides audio feedback to guide the user based on the learned map. The project builds upon a previous project B2G 1.0 -- TRAVELING for people with mobility impairmentsand outlines potential use of hardware from the SUPERBOX.I will refer to this project as B2G1.0 from the remainder of this document..

The SUPERBOX Contents

The superbox contains plenty of hardware to aid in the implementation of my idea.

The superbox contains:

However, I need to figure out what hardware to choose. I only need certain hardware components.

PRIVATE FEEDBACK FROM CONTEST MASTERS

For this contest challenge the organizers used Contest Masters to provide tailored feedback to my idea proposal survey that I filled out at the beginning of the contest..I received the feedback via an email.FEEDBACK SUMMARY report from the Contest Masters.

My initial proposal for this project was a continuation of my B2G 1.0 project, which involved developing a device to detect walls and obstacles for individuals with mobility impairments.

The feedback received from the Contest Masters placed emphasis on the necessity to prioritize wheelchair maneuverability, such as the incorporation of devices capable of facilitating wheelchair pivoting or sliding over obstacles, particularly in confined spaces. However, in light of my personal limited mechanical expertise, the focus will shift toward guiding wheelchair users to their intended destination while avoiding obstacles and constricted areas. Accordingly, the primary objectives have been revised to reflect this adjustment, and only the two operational modes, namely Learn Mode and Run Mode, will be retained. This prompted me to undertake a reassessment of my original concept and engage in introspective contemplation. Through this process of inquiry, a novel pathway emerged, imbued with fresh and fundamental objectives.

The main objectives of the project now are:

• Guiding wheelchair users to their desired location: The device will help users navigate unfamiliar spaces by providing audio feedback and directions.
• Avoiding obstacles and tight spaces: The device will map the environment and identify potential obstacles, allowing users to plan a safe and efficient route.
• Improving wheelchair maneuverability: While not directly manipulating the wheelchair, the device will focus on guiding users to avoid obstacles and tight spaces, indirectly improving maneuverability.

Specification

These functional and technical specifications serve as a blueprint for the development and evaluation of the proposed solution, ensuring that it meets the desired criteria for practicality, effectiveness, and user benefit. The specification includes some points from the B2G1.0 project.

Research Mobility in Unfamiliar Accommodations

I wanted to dig deeper into the problem by addressing some questions on implementing a mobility device for wheelchairs.

What are some potential challenges in implementing obstacle detection systems in hotel rooms and how can these challenges be addressed?

• Implementing obstacle detection systems in hotel rooms presents several potential challenges:
• Cost: The installation and maintenance of sensors and software can be expensive, potentially increasing the cost of hotel stays.
• Privacy: Guests may be uncomfortable with the idea of sensors monitoring their movements in a private space.
• Reliability: The system needs to be highly reliable to avoid false alarms or missed obstacles, which could lead to accidents.
• Complexity: Integrating the system with existing hotel infrastructure and ensuring compatibility with different room layouts and furniture arrangements can be complex.
• Aesthetics: The sensors and other components need to be discreet and not detract from the overall aesthetic of the room.

These challenges can be addressed through careful design and planning:

• Cost-effective solutions: Explore the use of affordable sensor technologies and open-source software to minimize costs.
• Privacy-preserving design: Implement data anonymization and allow guests to opt out of the system if they prefer.
• Rigorous testing: Conduct thorough testing in various room configurations to ensure the system's reliability.
• Modular design: Develop a modular system that can be easily adapted to different room layouts and furniture arrangements.
• Aesthetic integration: Design sensors and components that blend seamlessly with the room's décor.

Modular design addresses the challenge of varying room layouts and furniture arrangements by allowing for flexibility and customization. Here's how:

• Reconfiguration: Modular systems are composed of individual components or modules that can be easily rearranged, added, or removed. This means the system can be adapted to fit different room sizes, shapes, and furniture placements without major overhauls.
• Scalability: If a hotel room has a unique layout or requires additional coverage, more modules can simply be added to the system. This scalability ensures that the obstacle detection system can effectively cover the entire space, regardless of its configuration.
• Customization: Modular systems often offer a variety of sensor types and configurations. This allows hotels to choose the most appropriate sensors for their specific needs and room layouts. For example, some areas might require more sensitive or specialized sensors than others.
• Ease of Installation: Modular components are typically designed for easy installation, often with plug-and-play functionality. This simplifies the setup process and reduces the need for complex wiring or calibration.

By offering this flexibility, a modular design ensures that obstacle detection systems can be effectively deployed in a wide range of hotel rooms, regardless of their individual characteristics.

Here are some specific examples of modular components that can be used to create a flexible obstacle detection system in hotel rooms:

Sensor Modules:

• Ultrasonic distance sensors: These sensors emit sound waves and measure the time it takes for the waves to bounce back after hitting an object. They are good for detecting large obstacles but may have limited sensitivity for smaller objects.
• Infrared sensors: These sensors emit infrared light and detect the reflected light from objects. They are effective for detecting obstacles within a shorter range and can be more sensitive to smaller objects.
• LiDAR sensors: These sensors use laser light to create a 3D map of the environment. They offer high accuracy and can detect obstacles with varying shapes and sizes.
• Camera modules: Cameras can be used to visually detect obstacles and identify their type. However, they may require more processing power and raise privacy concerns.

Processing Modules:

• Microcontrollers: These small computers can process sensor data and make decisions about obstacle detection. They can be programmed with different algorithms to suit the specific needs of the system.
• Single-board computers: More powerful than microcontrollers, single-board computers like Raspberry Pi can handle more complex processing tasks, such as image recognition or machine learning.

Communication Modules:

• Wireless modules: These modules allow the system to communicate with other devices, such as a smartphone or a central control unit. They can use technologies like Wi-Fi, Bluetooth, or Zigbee.
• Wired modules: For more reliable and secure communication, wired modules can be used to connect the system to a power source or a central control unit.

User Interface Modules:

• Audio output modules: These modules can produce sound alerts to warn users about obstacles.
• Haptic feedback modules: These modules provide tactile feedback, such as vibrations, to alert users about obstacles.
• Visual display modules: These modules can display information about detected obstacles, such as their distance and location.

By combining these modular components in different configurations, a flexible and adaptable obstacle detection system can be created to suit the specific needs of different hotel rooms and users.

A modular obstacle detection system for hotel rooms could incorporate several additional features and functionalities to enhance its usability and effectiveness:

• Object Identification: The system could not only detect obstacles but also identify their type (e.g., furniture, luggage, or pets). This information could be used to provide more specific alerts or guidance to the user. For instance, if the system detects a pet, it could send a message to the user's smartphone with a reminder to keep the pet on a leash.
• Height Detection: In addition to horizontal obstacles, the system could also detect obstacles at different heights, such as low-hanging objects or raised thresholds. This would help prevent head injuries and improve overall safety. For example, the system could alert users to a low-hanging light fixture or a raised threshold in the bathroom.
• User Preferences: The system could be customizable to accommodate individual user preferences. For example, users could adjust the sensitivity of the sensors, the types of alerts, or the areas of the room to be monitored. This would allow users to tailor the system to their specific needs and preferences.
• Integration with Smart Devices: The system could be integrated with other smart devices in the room, such as voice assistants or smartphones. This would allow users to receive alerts through their preferred channels and control the system using voice commands or a mobile app. For instance, users could ask their voice assistant to turn on the obstacle detection system or check for obstacles in a specific area of the room.
• Data Logging and Analysis: The system could collect data on obstacle encounters and user behavior. This data could be analyzed to identify patterns and areas for improvement, helping to refine the system's performance and user experience. For example, the data could be used to identify areas of the room where obstacles are frequently encountered, allowing hotel staff to address these issues.
• Remote Monitoring and Support: Hotel staff could remotely monitor the system's performance and provide technical support if needed. This would ensure that the system remains operational and that any issues are addressed promptly. Remote monitoring would also allow hotel staff to proactively identify and resolve potential problems before they impact guests.

How could the system leverage artificial intelligence (AI) to improve its obstacle detection capabilities and provide personalized recommendations to users?

AI could be used to significantly enhance the capabilities of the obstacle detection system in several ways:

• Improved accuracy and adaptability: AI algorithms, particularly machine learning models, can be trained to recognize and classify obstacles with high accuracy. The system could learn to distinguish between different types of obstacles, such as furniture, luggage, or even people, and adapt to changes in the environment over time.
• Predictive obstacle avoidance: By analyzing sensor data in real-time, AI could anticipate the user's trajectory and predict potential collisions before they occur. This would allow the system to provide early warnings or suggest alternative routes to avoid obstacles.
• Personalized user experience: AI could learn from the user's behavior and preferences, tailoring its alerts and recommendations accordingly. For example, it could prioritize alerts for obstacles that are most relevant to the user's specific needs or mobility level.
• Continuous learning and improvement: The system could collect data on obstacle encounters and user interactions, using AI to analyze this data and identify areas for improvement. This would enable the system to continuously refine its algorithms and become more effective over time.

Here are some specific examples of how AI could be implemented:

• Object recognition: A convolutional neural network (CNN) could be trained to identify different types of obstacles from camera or LiDAR data.
• Trajectory prediction: A recurrent neural network (RNN) could be used to model the user's movement patterns and predict their future trajectory.
• Personalized recommendations: A reinforcement learning algorithm could be used to learn the user's preferences and provide personalized recommendations for obstacle avoidance.

By leveraging AI, the obstacle detection system could become more intelligent, adaptable, and user-friendly, significantly improving the safety and independence of people with mobility impairments

How could AI be used to integrate the obstacle detection system with other assistive technologies, such as smart wheelchairs or walking aids?

AI could play a crucial role in integrating the obstacle detection system with other assistive technologies, creating a more comprehensive and seamless experience for users with mobility impairments. Here's how:

• Data Fusion and Sensor Integration: AI algorithms could combine data from the obstacle detection system with data from other sensors on the wheelchair or walking aid, such as gyroscopes, accelerometers, and wheel encoders. This would create a more complete picture of the user's environment and movements, allowing for more accurate and timely obstacle avoidance maneuvers.
• Adaptive Control Strategies: AI could analyze the combined sensor data to develop adaptive control strategies for the wheelchair or walking aid. For example, it could adjust the speed, direction, or braking of the device to avoid obstacles while maintaining stability and comfort for the user.
• Personalized Assistance: AI could learn the user's individual needs and preferences, tailoring the assistive technology's behavior accordingly. For instance, it could prioritize obstacle avoidance in certain areas of the room or adjust the level of assistance based on the user's fatigue or confidence level.
• Proactive Guidance: By predicting the user's intended path and potential obstacles, AI could provide proactive guidance and suggestions. This could include suggesting alternative routes, highlighting potential hazards, or offering step-by-step navigation instructions.
• Seamless Communication: AI could facilitate communication between the user and the assistive technology through natural language processing or other intuitive interfaces. This would allow users to easily control the device, provide feedback, and receive personalized assistance.

By integrating AI into the obstacle detection system and other assistive technologies, a more intelligent and responsive ecosystem of support could be created for people with mobility impairments, enhancing their safety, independence, and overall quality of life.

RESEARCH CONCLUSIONS

This project objective is to develop a device that facilitates the navigation of wheelchair users within unfamiliar environments, enabling them to reach their desired destinations while circumventing obstacles and constricted spaces. During the research phase, mentioned in the previous section, I have reached the following conclusions:

The device will have two modes of operation:

• Learn Mode: In this mode, the wheelchair will map the environment and create a detailed map.
• Run Mode: In this mode, the wheelchair will use the map it created in Learn mode to provide audio feedback to the user, guiding them through the environment.

Additional Features:

• Route Planning: In RUN mode, the wheelchair can use its map to plan the best route from point A to point B.
• Obstacle Detection: If the wheelchair detects an obstacle in its path, it will re-route and send a message to the user informing them of the obstacle.
• Size of Wheelchair: The wheelchair can take into account its own size when planning routes, ensuring that it can safely navigate through tight spaces.

Use Camera and AI mapping:

• In LEARN mode, the wheelchair would use its camera to capture images of its surroundings. These images would then be processed by an AI algorithm to create a map of the environment. This map would include information about the location of objects, obstacles, and other relevant features.
• In RUN mode, the wheelchair would use the map created in LEARN mode to navigate its environment. The wheelchair would use the camera to track its position relative to the map and would use this information to avoid obstacles and reach its destination.

Use other sensors on the wheelchair (Accelerometers and Gyroscopes or LDAR)) to aid in Object detection:

• Accelerometers and gyroscopes can be used to measure the wheelchair's movement and orientation. This information can be used to detect objects that are moving or that are in the wheelchair's path.
• LDAR (Light Detection and Ranging) is a sensor that can be used to measure the distance between the wheelchair and objects in its environment. This information can be used to create a more detailed map of the environment and to detect objects that are not visible to the camera.
• By combining the information from the camera, accelerometers, gyroscopes, and LDAR, the wheelchair can create a more comprehensive understanding of its environment and navigate more safely and efficiently.

Use the DFRobot UniHiker from the SUPERBOX:

• Use it as the center of the system:
• The UniHiker can serve as the central hub of your system, connecting all of the different components and devices.
• It provides a stable and reliable platform for running your applications and software.
• The UniHiker's powerful processor and ample memory ensure that it can handle even the most demanding tasks.
• Use the screen to display the interface to the system:
• The UniHiker's built-in screen can be used to display the user interface for your system.
• This allows users to interact with the system and control its various functions.
• The screen can also be used to display information and data from the system's sensors and devices.
• Attach the camera module and audio devices to it:
• The UniHiker can be equipped with a variety of camera modules and audio devices.
• These devices can be used to capture images and videos, as well as record and playback audio.
• The UniHiker's built-in microphone and speaker can also be used for audio input and output.

Additional tips for using the UniHiker:

• Use the UniHiker's built-in Wi-Fi and Bluetooth connectivity to connect to other devices and networks.
• Use the UniHiker's GPIO pins to connect to external sensors and devices.
• Use the UniHiker's onboard sensors, such as the accelerometer and gyroscope, to detect motion and orientation.
• Use the UniHiker's built-in battery to power the system for extended periods of time.

The project will incorporate AI to improve obstacle detection and provide personalized recommendations and will explore integration with other assistive technologies. Specific examples include:

• Object recognition using convolutional neural networks (CNNs) to identify obstacles from camera or LiDAR data.
• Trajectory prediction using recurrent neural networks (RNNs) to model user movement patterns and predict future trajectories.
• Personalized recommendations using reinforcement learning algorithms to learn user preferences and provide customized obstacle avoidance suggestions.

AI can also be utilized to enhance the obstacle detection system and other assistive technologies by:

• Integrating data from various sensors, developing adaptive control strategies, offering personalized assistance, providing proactive guidance, and enabling seamless communication. This integration allows for a more comprehensive and tailored assistive ecosystem, improving the safety, independence, and quality of life for individuals with mobility impairments.

Use a Modular design approach

• Modular design is a design strategy that splits a system into smaller parts called modules, which can be independently developed, modified, replaced, or exchanged.
• A modular obstacle detection system for hotel rooms can be enhanced with various features and functionalities to improve its usability and effectiveness. These features include object identification, height detection, user preferences, integration with smart devices, data logging and analysis, and remote monitoring and support. By incorporating these features, the system can provide more specific alerts, prevent injuries, accommodate individual user needs, and allow for remote monitoring and support, ultimately enhancing the guest experience and improving hotel operations.

Be aware of Potential Challenges

• Potential challenges in implementing such a system include cost, privacy concerns, reliability, complexity, and aesthetics.

Functional and technical specifications are critical components in the software development process, serving distinct but complementary purposes.

The functional and technical specifications serve as a blueprint for the development and evaluation of the proposed solution, ensuring that it meets the desired criteria for practicality, effectiveness, and user benefit.

• A functional specification outlines sensor requirements, operating modes, audio feedback, user interface, and power management strategies.
• A technical specification details hardware components, software design, communication protocols, performance metrics, and testing and validation procedures.

Specification

These functional and technical specifications serve as a blueprint for the development and evaluation of the proposed solution, ensuring that it meets the desired criteria for practicality, effectiveness, and user benefit. The specification includes some points from the B2G1.0 project.

Functional Specification

Here's a functional specification based on the hardware available in the Superbox:

Core Components:

Microcontroller: Swan v3 (STM32L4+): Offers a good balance of power, memory, and peripherals.

Wireless Communication: Blues Notecard WiFi: Provides necessary connectivity for data logging and potential future features.

Single Board Computer: DFRobot UNIHIKER: Is utilized as a Single Board Computer, and is the heart of the system

Camera Module: Seeed Studio Grove - Vision AI Module: Compact and offers pre-trained models for object detection, potentially simplifying development. The OV7670 Camera will be used to capture images of the environment.

MAX32666FTHR Board Accelerometer and Gyroscope: This sensor on the board, will be used to measure the wheelchair's acceleration and angular velocity, aiding in movement detection and orientation.

Power Supply: Battery: Utilize the Notecarrier-A's battery holder for a rechargeable LiPo battery. This provides a compact and integrated power solution.

User Interface: use the DFRobot UNIHIKER buttons and displa for mode selection, configuration, and user feedback.

Assembly:

Mount components: Securely attach the Swan v3, Notecarrier-A, MAX32666FTHR within an enclosure using appropriate mounting hardware or adhesive.

Connect camera: Connect the Vision AI Module to the Swan v3 using the Grove connector.

Wiring: Carefully connect the Swan v3's and the MAX32666FTHR to the Unihiker GPIO pins, ensuring correct polarity.

Battery: Insert the rechargeable LiPo battery into the Notecarrier-A's battery holder.

Enclosure: Seal the enclosure, ensuring all openings are properly closed and waterproof.

Operating Modes:

Learn Mode: In this mode, the wheelchair will move through the environment, capturing images and sensor data. The system will process this data to create a map of the environment.

Run Mode: In this mode, the wheelchair will use the map created in Learn Mode to navigate to a desired location, avoiding obstacles and tight spaces.

Audio Feedback:

Audio Module: The system will use the onboard speaker on the DFRobot Unihiker to provide audio feedback to the user. This will include:

Directional cues: Indicating turns (e.g., "turn left," "turn right").

Obstacle warnings: Alerting the user to obstacles in their path (e.g., "obstacle ahead").

Confirmation sounds: Confirming user inputs or successful actions (e.g., "destination set").

User Interface:

LCD Screen: The system will utilize the Unihiker's LCD screen to display a basic user interface. This interface will allow the user to:

Select mode: Switch between Learn Mode and Run Mode.

Set destination: Input the desired destination within the mapped environment.

View map: Display a simplified representation of the mapped environment.

Adjust settings: Modify system parameters like sensitivity or volume.

Power Management:

Lithium Battery: The system will be powered by a 3.7V Lithium battery.

Power-saving features: Implement power-saving strategies like:

Screen dimming: Automatically dim the screen after a period of inactivity.

Sleep mode: Enter a low-power sleep mode when not in use.

Optimized algorithms: Design algorithms to minimize power consumption during sensor data processing.

Technical Specification

Here's a technical specification based on the hardware mentioned in the Core Components sectionabove:

Microcontroller:

Model: Swan v3 (STM32L4+)

Specifications:

ARM Cortex-M4 core with 120 MHz clock speed

1 MB Flash memory, 320 KB RAM

Multiple communication interfaces (I2C, SPI, UART, USB)

Low-power consumption modes

Wireless Communication:

Model: Blues Notecard WiFi

Specifications:

Wi-Fi connectivity (802.11 b/g/n)

Secure data transmission

Cloud integration for data logging and remote management

Single Board Computer:

Model: DFRobot UNIHIKER

Specifications:

ESP32-based microcontroller with dual-core processor

4 MB Flash memory, 520 KB RAM

Built-in 2.8" LCD touch screen

Multiple communication interfaces (I2C, SPI, UART, USB, Wi-Fi, Bluetooth)

Camera Module:

Model: Seeed Studio Grove - Vision AI Module

Specifications:

13 MP image sensor

Pre-trained models for object detection and recognition

Grove connector for easy integration

Accelerometer and Gyroscope:

Model: MAX32666FTHR Board

Specifications:

3-axis accelerometer and 3-axis gyroscope

High precision and low noise

I2C communication interface

Power Supply:

Battery: Rechargeable LiPo battery

Battery Holder: Notecarrier-A's battery holder for compact integration

BUILD

Here's a well-described build process based on the functional and technical specifications:

1. Component Assembly:

Microcontroller and Wireless Communication:

Securely mount the Swan v3 microcontroller and Blues Notecard WiFi module onto the Notecarrier-A board using standoffs and screws, ensuring firm connections.

Single Board Computer:

Attach the DFRobot UNIHIKER board to a suitable enclosure or mounting plate using appropriate hardware, ensuring accessibility to ports and buttons.

Camera Module:

Connect the Seeed Studio Grove - Vision AI Module to the Swan v3 microcontroller using the Grove connector, ensuring proper alignment.

Accelerometer and Gyroscope:

Carefully connect the MAX32666FTHR Board to the Swan v3 microcontroller's I2C pins, ensuring correct polarity and secure connections.

Power Supply:

Insert the rechargeable LiPo battery into the Notecarrier-A's battery holder, ensuring a snug fit.

2. Wiring and Connections:

Power Connections:

Connect the LiPo battery's power leads to the appropriate power input pins on the Swan v3 microcontroller, observing correct polarity.

Communication Connections:

Connect the Swan v3 microcontroller's UART pins to the Blues Notecard WiFi module's UART pins, enabling serial communication.

Connect the Swan v3 microcontroller's I2C pins to the DFRobot UNIHIKER's I2C pins, allowing for data exchange and control.

Camera Connection:

Ensure the camera module is securely connected to the Swan v3 microcontroller's Grove connector.

Accelerometer and Gyroscope Connection:

Verify the I2C connection between the MAX32666FTHR Board and the Swan v3 microcontroller.

3. Enclosure and Protection:

Enclosure Selection:

Choose an appropriate enclosure that accommodates all components, provides necessary protection, and allows for easy access to ports and buttons.

Component Placement:

Carefully position all components within the enclosure, ensuring they are securely mounted and do not interfere with each other.

Cable Management:

Neatly arrange and secure all cables and wires within the enclosure to prevent accidental disconnections or damage.

Environmental Protection:

Consider adding weatherproofing or dustproofing measures if the device will be used in outdoor or harsh environments.

4. Software Integration:

Microcontroller Programming:

Develop and upload firmware to the Swan v3 microcontroller to handle sensor data acquisition, processing, and communication with other components.

Wireless Communication Setup:

Configure the Blues Notecard WiFi module for network connectivity and data transmission.

Single Board Computer Programming:

Program the DFRobot UNIHIKER to provide the user interface, display sensor data, and control the wheelchair's navigation system.

Sensor Calibration:

Calibrate the camera module, accelerometer, and gyroscope to ensure accurate data readings.

5. Testing and Validation:

Functional Testing:

Test each component and connection individually to verify proper operation.

Test the integrated system in both Learn Mode and Run Mode to ensure it meets the functional requirements.

Performance Testing:

Evaluate the system's performance in terms of accuracy, responsiveness, and power consumption.

User Acceptance Testing:

Conduct user testing to gather feedback and identify areas for improvement.

IMPLEMENTATION

This section contains implementation details.

Here's a breakdown of the implementation details, focusing on the key hardware and software components:

Hardware:

Microcontroller (Swan v3):

Sensor Integration: The microcontroller will be programmed to interface with the camera module, accelerometer, and gyroscope. It will handle tasks like capturing images, reading sensor data, and processing this information to make real-time decisions.

Communication: The microcontroller will communicate with the Blues Notecard WiFi module to send sensor data and receive commands. It will also communicate with the DFRobot Unihiker to display information and receive user input.

Wireless Communication (Blues Notecard WiFi):

Data Transmission: This module will be responsible for transmitting sensor data and other information to a cloud-based platform or a local server. This data can be used for further analysis, remote monitoring, or integration with other systems.

Connectivity: The module will handle the WiFi connection, ensuring that the wheelchair remains connected to the network for continuous data transmission and remote management.

Single Board Computer (DFRobot Unihiker):

User Interface: The Unihiker's LCD screen and buttons will be used to provide a user-friendly interface. This interface will allow the user to switch between modes, set destinations, view the map, and adjust settings.

Audio Feedback: The Unihiker's onboard speaker will be used to provide audio cues and warnings to the user, guiding them through the environment and alerting them to obstacles.

Camera Module (Seeed Studio Grove - Vision AI Module):

Image Capture: The camera module will capture images of the environment, which will be used for mapping and object detection.

Object Detection: The module's pre-trained models will be used to detect and identify obstacles in the captured images. This information will be used to guide the wheelchair's navigation.

Accelerometer and Gyroscope (MAX32666FTHR Board):

Movement Detection: These sensors will measure the wheelchair's acceleration and angular velocity. This data will be used to detect movement, determine the wheelchair's orientation, and assist in navigation.

Software:

Microcontroller Firmware:

Sensor Data Acquisition: The firmware will be responsible for collecting data from the camera, accelerometer, and gyroscope.

Data Processing: The firmware will process the sensor data to create a map of the environment, detect obstacles, and make navigation decisions.

Communication: The firmware will handle communication with the Blues Notecard WiFi module and the DFRobot Unihiker.

DFRobot Unihiker Software:

User Interface: The software will provide a user-friendly interface for interacting with the system.

Audio Feedback: The software will generate audio cues and warnings based on the sensor data and navigation decisions.

Cloud-Based Platform (or Local Server):

Data Storage: The platform will store the sensor data and other information transmitted from the wheelchair.

Data Analysis: The platform can be used to analyze the collected data to gain insights into user behavior, system performance, and potential areas for improvement.

Remote Management: The platform can be used to remotely monitor and manage the wheelchair's navigation system.

Implementation Challenges and Considerations:

Sensor Fusion: Combining data from multiple sensors (camera, accelerometer, gyroscope) can be challenging. Careful calibration and sensor fusion algorithms will be needed to ensure accurate and reliable obstacle detection and navigation.

Power Consumption: The system needs to be designed for low power consumption to maximize battery life. This will involve optimizing algorithms, implementing power-saving modes, and carefully selecting hardware components.

Real-time Performance: The system needs to operate in real-time to provide timely feedback and guidance to the user. This will require efficient algorithms and hardware that can handle the computational demands of sensor data processing and navigation.

User Experience: The user interface needs to be intuitive and easy to use, even for users with limited technical expertise. The audio feedback should be clear and informative, providing the user with the necessary information to navigate safely.

RUNNING THE SYSTEM

Here's a detailed description of how to run the Smart Wheelchair Navigation System:

Startup:

Power On: Turn on the DFRobot Unihiker board. The system will perform a self-check and initialize all sensors and components.

Mode Selection: The Unihiker's LCD will display the main menu, allowing the user to select between "Learn Mode" and "Run Mode."

Learn Mode:

Initiate Mapping: Select "Learn Mode" on the Unihiker's display. The system will prompt the user to start moving the wheelchair around the environment.

Data Collection: As the wheelchair moves, the camera module will capture images, and the accelerometer and gyroscope will record movement data. The microcontroller will process this data to create a map of the environment.

Mapping Progress: The Unihiker's display will show a progress bar or other visual indication of the mapping process. The user can continue moving the wheelchair until the desired area is mapped.

Save Map: Once the mapping is complete, the user can save the map to the system's memory or a cloud-based platform.

Run Mode:

Load Map: Select "Run Mode" on the Unihiker's display. The system will prompt the user to load a previously saved map.

Set Destination: Once the map is loaded, the user can set their desired destination on the Unihiker's display. This can be done by selecting a point on the map or by entering coordinates.

Navigation: The system will calculate the optimal route to the destination, taking into account obstacles and tight spaces. The wheelchair will then start moving towards the destination.

Audio Guidance: As the wheelchair moves, the system will provide audio feedback to the user, indicating turns, obstacles, and other relevant information.

Obstacle Avoidance: If the wheelchair encounters an obstacle, the system will detect it using the camera module and sensor data. It will then re-route the wheelchair to avoid the obstacle and continue towards the destination.

Arrival: When the wheelchair reaches the destination, the system will provide an audio notification to the user.

Additional Features:

User Profiles: Users can create profiles to store their preferences for route planning, obstacle avoidance, and terrain preferences.

Real-time Feedback: The system can gather feedback from the user during navigation to improve its understanding of their preferences.

Machine Learning: The system can use machine learning algorithms to analyze user behavior and provide more personalized recommendations.

CONCLUSIONS

The Smart Wheelchair Navigation System is a device designed to assist individuals with mobility impairments in navigating unfamiliar environments. The device operates in two modes: Learn Mode, where it maps the environment using sensors and AI, and Run Mode, where it provides audio feedback to guide the user based on the learned map.

The project aims to address challenges faced by wheelchair users in navigating obstacles and tight spaces, particularly in hotel rooms. It utilizes a modular design approach, incorporating various sensors and AI algorithms to enhance its functionality and adaptability.

The system's core components include a microcontroller, wireless communication module, single-board computer, camera module, accelerometer, gyroscope, and power supply. These components work together to collect and process sensor data, create a map of the environment, detect obstacles, and provide audio feedback to the user.

The implementation details outline the hardware and software components, highlighting their specific roles in the system's operation. The document also discusses potential challenges and considerations, such as sensor fusion, power consumption, real-time performance, and user experience.

The Smart Wheelchair Navigation System offers features like route planning, obstacle detection, and personalized recommendations based on user preferences and behavior patterns. The system utilizes machine learning algorithms to continuously learn and adapt to individual user needs, enhancing its accuracy and effectiveness over time.

So, the Design Phase took longer than I thought, and the contest deadline is September 4, 2014, which is creeping up fast. I'm not quite finished with the project yet, but I'm working hard to wrap it up. I'm really excited about my project and I'm confident that it will be a valuable resource for others. I plan to document every step of the way, so people can see how I did it and maybe even replicate it themselves. Once I'm finished, I'll be showcasing a fully functional prototype, along with Code, Schematics and Instructions on how to operate it. This will give people a chance to see what the project can do and how it works. I'm also going to be seeking feedback from users and experts to help me refine and improve the prototype. I want to make sure it's the best it can be and meets the highest standards of quality and functionality. I'm really excited to share this project with the world and I hope it will make a positive contribution to the field.

ENHANCEMENTS

Here are some additional features that could enhance the functionality of the wheelchair navigation system:

Route Planning: In RUN mode, the wheelchair could use its map to plan the best route from point A to point B, taking into account factors like distance, obstacles, and user preferences.

Obstacle Detection: If the wheelchair detects an obstacle in its path, it could re-route and send a message to the user informing them of the obstacle.

Size of Wheelchair: The wheelchair could take into account its own size when planning routes, ensuring that it can safely navigate through tight spaces.

The wheelchair's route planning could be enhanced to take into account the user's preferences in several ways:

User Profiles: The system could allow users to create profiles where they specify their preferences, such as:

Preferred routes: Users could indicate if they prefer shorter routes, even if they are more complex, or if they prefer simpler routes, even if they are slightly longer.

Obstacle avoidance: Users could specify their comfort level with navigating near obstacles. For example, a user who is more cautious might prefer a route that keeps a wider berth from obstacles.

Terrain preferences: Users could indicate if they prefer to avoid certain types of terrain, such as inclines or uneven surfaces.

Real-time Feedback: The system could gather feedback from the user during navigation to further refine its understanding of their preferences. For example, if the user frequently deviates from the suggested route, the system could learn that they prefer a different type of route.

Machine Learning: The system could use machine learning algorithms to analyze the user's past behavior and preferences to make more accurate predictions about their future preferences. This would allow the system to proactively suggest routes that are most likely to be preferred by the user.

To enhance the wheelchair's route planning for more personalized recommendations, consider these strategies:

User Profiles: Allow users to input preferences like preferred routes (shortest, simplest, etc.), obstacle avoidance comfort levels, and terrain preferences.

Real-time Feedback: Gather user input during navigation to understand their preferences better. If they often deviate from suggested routes, learn from those choices.

Machine Learning: Analyze past user behavior and preferences using machine learning algorithms to predict future needs and proactively suggest routes.

Here's how the system could use machine learning algorithms to analyze user preferences and provide more accurate recommendations:

Data Collection: The system would continuously gather data on user interactions, including:

Chosen routes: The actual paths taken by the user, including any deviations from suggested routes.

Obstacle responses: How the user reacts to obstacles, such as whether they follow the suggested rerouting or find alternative paths.

User feedback: Explicit feedback from the user, such as ratings of suggested routes or obstacle avoidance maneuvers.

Feature Extraction: Relevant features would be extracted from the collected data, such as:

Route characteristics: Length, complexity, number of turns, elevation changes.

Obstacle types: Size, shape, location.

User behavior patterns: Tendency to choose shorter/simpler routes, obstacle avoidance preferences.

Model Training: Machine learning algorithms, such as decision trees, support vector machines, or neural networks, would be trained on the extracted features and associated user preferences. The goal is to learn a model that can accurately predict user preferences based on new data.

Preference Prediction: The trained model would be used to predict the user's preferences for new routes or obstacle avoidance scenarios. For example, the model could predict the user's likelihood of choosing a particular route based on its characteristics and the user's past behavior.

Personalized Recommendations: The system would use the predicted preferences to provide personalized recommendations to the user. For example, it could suggest routes that are most likely to be preferred by the user, or it could adjust obstacle avoidance strategies based on the user's comfort level.

Continuous Learning: The system would continuously update its models as new data becomes available, allowing it to adapt to changes in the user's preferences over time. This would ensure that the recommendations remain accurate and relevant

Thank you

to the Hackster community for taking the time to read through my project. I hope that you found the information enlightening and that it sparked some ideas for future assistive technology developments. Your feedback and support are invaluable as we work together to create a more inclusive world for individuals with mobility impairments.