AudioGuide - AI-Powered Environmental Navigator

AudioGuide - AI-Powered Environmental Navigator

Story

For millions of people with visual impairments, navigating outdoor environments presents daily challenges. While traditional aids like white canes and guide dogs provide invaluable assistance, modern technology offers opportunities to enhance outdoor navigation and independence. AudioGuide aims to bridge this gap by creating an AI-powered environmental navigation system using the DFRobot UniHiker board.

The Problem

People with visual impairments face several challenges when navigating outdoor environments:

• Difficulty identifying obstacles and hazards in real-time
• Limited ability to read signs and text in the environment
• Challenges in understanding spatial relationships and distances
• Reduced confidence in exploring new areas independently

The Solution

AudioGuide transforms the powerful UniHiker board into an intelligent navigation assistant. By combining computer vision, AI processing, and intuitive audio feedback, it provides real-time environmental information to users, enhancing their spatial awareness and confidence in outdoor navigation.

Software

Dependencies:

python

python

Copy

import cv2
import numpy as np
import tensorflow as tf
import gtts
import pygame
import serial
import time
import RPi.GPIO as GPIO

Key Software Components:

• Object Detection System:

python

python

Copy

def detect_objects(frame):
    # Convert frame to blob for neural network
    blob = cv2.dnn.blobFromImage(frame, 1/255.0, (416, 416), swapRB=True, crop=False)
    net.setInput(blob)
    
    # Get detections
    layer_outputs = net.forward(output_layers)
    
    # Process detections
    class_ids = []
    confidences = []
    boxes = []
    
    for output in layer_outputs:
        for detection in output:
            scores = detection[5:]
            class_id = np.argmax(scores)
            confidence = scores[class_id]
            
            if confidence > 0.5:
                # Process detection coordinates
                center_x = int(detection[0] * width)
                center_y = int(detection[1] * height)
                w = int(detection[2] * width)
                h = int(detection[3] * height)
                
                # Rectangle coordinates
                x = int(center_x - w / 2)
                y = int(center_y - h / 2)
                
                boxes.append([x, y, w, h])
                confidences.append(float(confidence))
                class_ids.append(class_id)
    
    return boxes, class_ids, confidences

• Audio Feedback System:

python

python

Copy

class AudioFeedback:
    def __init__(self):
        pygame.mixer.init()
        self.audio_cache = {}
        
    def generate_audio(self, text):
        if text not in self.audio_cache:
            tts = gtts.gTTS(text)
            temp_file = f"temp_{hash(text)}.mp3"
            tts.save(temp_file)
            self.audio_cache[text] = temp_file
        return self.audio_cache[text]
    
    def play_audio(self, text):
        audio_file = self.generate_audio(text)
        pygame.mixer.music.load(audio_file)
        pygame.mixer.music.play()
        while pygame.mixer.music.get_busy():
            continue

• GPS Navigation:

pytho

python

Copy

class GPSNavigator:
    def __init__(self, port='/dev/ttyUSB0'):
        self.ser = serial.Serial(port, 9600, timeout=0.5)
        
    def get_location(self):
        while True:
            data = self.ser.readline().decode('ascii', errors='replace')
            if data.startswith('$GPGGA'):
                parts = data.split(',')
                if len(parts) >= 6 and parts[2] and parts[4]:
                    lat = float(parts[2][:2]) + float(parts[2][2:]) / 60
                    lon = float(parts[4][:3]) + float(parts[4][3:]) / 60
                    return lat, lon
        return None, None

Build Instructions

Hardware Assembly:

• Mount the camera module to the UniHiker board
• Connect GPS module to UART pins
• Wire ultrasonic sensors to GPIO pins
• Install emergency button
• 3D print the case and assemble components
• Hardware Assembly:
  Mount the camera module to the UniHiker board
  Connect GPS module to UART pins
  Wire ultrasonic sensors to GPIO pins
  Install emergency button
  3D print the case and assemble components

Software Setup:

• Install required Python packages
• Configure system autostart
• Set up error logging
• Calibrate sensors
• Software Setup:
  Install required Python packages
  Configure system autostart
  Set up error logging
  Calibrate sensors

Testing:

• Verify object detection accuracy
• Test audio feedback clarity
• Validate GPS accuracy
• Check emergency button functionality
• Perform battery life testing
• Testing:
  Verify object detection accuracy
  Test audio feedback clarity
  Validate GPS accuracy
  Check emergency button functionality
  Perform battery life testing

Usage Instructions

• Power on the device using the power button
• Wait for the startup sound confirmation
• Put on the bone conduction headphones

The system will automatically begin:

• Detecting objects and obstacles
• Providing audio feedback
• Tracking location
• Monitoring for emergency button press
• The system will automatically begin:
  Detecting objects and obstacles
  Providing audio feedback
  Tracking location
  Monitoring for emergency button press

Audio Feedback Meanings:

• Single beep: Object detected
• Double beep: Obstacle in path
• Triple beep: Emergency mode activated
• Voice descriptions: Environmental information

Future Improvements

Enhanced Features:

• Machine learning for improved object recognition
• Integration with mapping services
• Crowd-sourced safe path database
• Weather API integration
• Enhanced Features:
  Machine learning for improved object recognition
  Integration with mapping services
  Crowd-sourced safe path database
  Weather API integration

Hardware Optimizations:

• Reduced power consumption
• Smaller form factor
• Waterproof casing
• Extended battery life
• Hardware Optimizations:
  Reduced power consumption
  Smaller form factor
  Waterproof casing
  Extended battery life

Software Enhancements:

• Personalized audio feedback
• Offline operation mode
• Enhanced emergency features
• Social features for community support
• Software Enhancements:
  Personalized audio feedback
  Offline operation mode
  Enhanced emergency features
  Social features for community support

Impact

AudioGuide demonstrates how accessible technology can enhance independence and confidence for people with visual impairments. The project:

• Provides real-time environmental awareness
• Enhances safety during outdoor navigation
• Increases independence in daily activities
• Creates opportunities for social interaction
• Builds confidence in exploring new environments

Acknowledgments

Special thanks to:

• DFRobot for the UniHiker board
• The visual impairment community for valuable feedback
• Open-source contributors to the libraries used
• BUILD2GETHER 2.0 for the opportunity to develop this solution