Created September 4, 2024

Indoor Information Accessibility System

Indoor Information Accessibility System for Visually Impaired Individuals

Things used in this project

Hardware components

Blues Notecard (Cellular)

Nordic Semiconductor nRF52 Development Kit

Tria Technologies Dialog SmartBond™ DA14586 Bluetooth 5 SoC

Seeed Studio XIAO ESP32S3 Sense

Seeed Studio Grove Shield for Seeeduino XIAO - with embedded battery management chip

Story

Indoor Information Accessibility System for the Visually Impaired - Overview of the Project
The scope of this project involves the design and implementation of a system for assisting a visually impaired person in reading indoor information, such as menu cards or display descriptions. This shall be achieved by converting the available visual materials into audio. The main hardware components for implementing the development system are a cellular-enabled device for transmitting text data, and a microcontroller with Bluetooth capability that will process and convert this text data into speech.

Components and Tools
Blues' Notecard Cellular NBGL

Provide cellular connectivity to send and receive data.
Interfaces with the Nordic Semiconductor nRF52840 DK board for sending and receiving text data input.
Nordic Semiconductor nRF52840 DK

This acts like the brain, performing some tasks related to Bluetooth and TTS.
It interfaces with the Notecard to convert the text it receives into speech that one can hear.
Development Environment

Arduino IDE: Used to create and upload code to the Notecard and Nordic DK.
Bluetooth Serial Library: Bluetooth communication on the Nordic DK
Notecard Library: Interfacing with the Blues' Notecard
Project Setup and Hardware Connections
Hardware Configuration
Notecard Cellular NBGL:
Connect to the Nordic nRF52840 DK using I2C by utilizing SDA and SCL lines.
The Notecard must be powered properly and configured for cellular connectivity.
nRF52840 DK:
Bluetooth module connected with the Nordic DK.
The Nordic DK needs to be configured to deal with Bluetooth communication and TTS functionality.

Explanation:

("nRF52840_TTS");: Configures Bluetooth communication with a name that identifies the Nordic DK.
sendToTTS(): Sends the text received across to any TTS module or app.
Project Flow
Setup:

Notecard: Your Notecard is already set up for cellular and should be able to send text data.
nRF52840 DK: Your Nordic DK is ready to go with some new code handling Bluetooth, as well as any text-to-speech functionality.
Data Exchange:

Notecard: The Notecard sends the text data over cellular to a host server or directly to the Nordic DK.
Data Reception and Processing:

nRF52840 DK: Receive text data over Bluetooth and perform the processing.
Text-to-Speech:

Conversion: The Nordic DK will either interface directly with a TTS module or send the text to a smartphone app to synthesize speech.
Testing and Calibration
Connectivity:

Verify the Notecard's cellular connectivity; it should be able to send and receive data with no issues.
Verify that Bluetooth communication can occur without any hitches between the Nordic DK and whatever peripheral is attached.
Receiving Data:

Verify that the text data is appropriately received by the Nordic DK and reflects on the serial monitor.

Text-to-Speech Output:

Legible Speech

The speech generated in the TTS application should be clear and legible to the listener.
More adjustment might be needed for better voices in TTS.
Future Development
Incorporate OCR:
Attach an OCR module where texts from cameras or scanners can be captured visually.
TTS Advanced Features:
Relay for more realistic speech in human voice,
User Interface:

User-friendly interactions that enable the visually impaired users to interact with the system better.
Better Connectivity:

Other modes of communications such as Wi-Fi or Direct Connections could be tested in order to enhance stability of the system and efficiency. Conclusion
It is a basic system to convert visual indoor information into an audio format for better accessibility by the visually impaired. The proposed system would be able to transmit, receive, and process textual data with the support of cellular and Bluetooth technologies in speech conversion. Further enhancement can be done to increase the capabilities of the system and widen the applicability of the system.

#include <nRF52840.h>
#include <BluetoothSerial.h>

BluetoothSerial SerialBT;

void setup() {
  Serial.begin(115200);
  SerialBT.begin("nRF52840_TTS");
  Serial.println("Bluetooth TTS started");
}

void loop() {
  if (Serial.available()) {
    String receivedText = Serial.readStringUntil('\n');
    Serial.println("Received text: " + receivedText);
    
    // Send text to TTS module or smartphone app
    sendToTTS(receivedText);
  }
}

void sendToTTS(String text) {
  // Implement TTS sending logic here.
  SerialBT.print(text);
}

#include <Wire.h>
#include <Notecard.h>

const char *NOTECARD_I2C_ADDRESS = "0x55";  // I2C address for Notecard
Notecard note;

void setup() {
  Serial.begin(115200);
  Wire.begin();
  
  if (!note.begin(NOTECARD_I2C_ADDRESS)) {
    Serial.println("Failed to initialize Notecard");
    while (1);  // Halt if initialization fails
  }
  
  // Example: Configure Notecard for cellular communication
  note.sendCmd(Serial, "card.network");
  note.sendCmd(Serial, "card.location");
}

void loop() {
  // Example: Send text to a server
  sendText("Example Text: Welcome to the exhibit.");
  delay(10000);  // Send data every 10 seconds
}

void sendText(const char* text) {
  char cmd[256];
  snprintf(cmd, sizeof(cmd), "{\"type\":\"note\",\"body\":{\"text\":\"%s\"}}", text);
  note.sendCmd(Serial, cmd);
}

Indoor Information Accessibility System

Things used in this project

Hardware components

Story

Code

nRF52840 DK Code

Notecard Initialization Code

Credits

Mithya

Comments

Embed the widget on your own site

Indoor Information Accessibility System

Indoor Information Accessibility System

Things used in this project

Hardware components

Story

Code

nRF52840 DK Code

Notecard Initialization Code

Credits

Mithya

Comments