Intro: Why this Project
Video Demonstration
Hardware Build
3D Printing or Improvisation
Programming
How to Use it for Learning the Braille
Application Areas
Conclusion
References

Published September 5, 2024 © CC BY

Braille Trainer Board: Learning Dots with Auditory Feedback

Learning the Braille Dots representing 26 English Alphabets with Audio Feedback for Visually Impaired Children

IntermediateFull instructions provided12 hours156

Lucky Draw for TWO Submissions

Build2gether 2.0 — Inclusive Innovation Challenge

Braille Trainer Board: Learning Dots with Auditory Feedback

Things used in this project

Hardware components

Sony Spresense boards (main & extension)

DFRobot Gravity:Digital Push Button (Yellow)

DFRobot Stereo Enclosed Speaker - 3W 8Ω

Flash Memory Card, MicroSD Card

Headphones, 20 Hz

Male/Female Jumper Wires

Seeed Studio Polymer Lithium Ion Battery - 2200mAh 3.7V

Rocker Switch, VISI-ROCKER

Software apps and online services

Arduino IDE

Hand tools and fabrication machines

Soldering iron (generic)

Story

Intro: Why this Project ?

Learning Braille can be challenging for some visually impaired children, even some grown ups might struggle to map the dots in their mind.

Self Learning with Sound Feedback seems like a great way to learn Braille

From Wikipedia Braille wiki

"Children who are blind miss out on fundamental parts of early and advanced education if not provided with the necessary tools, such as access to educational materials in braille. Children who are blind or visually impaired can begin learning foundational braille skills from a very young age to become fluent braille readers as they get older. Sighted children are naturally exposed to written language on signs, on TV and in the books they see. Blind children require the same early exposure to literacy, through access to braille rich environments and opportunities to explore the world around them. Print-braille books, for example, present text in both print and braille and can be read by sighted parents to blind children (and vice versa), allowing blind children to develop an early love for reading even before formal reading instruction begins.

Adults who experience sight loss later in life or who did not have the opportunity to learn it when they were younger can also learn braille. In most cases, adults who learn braille were already literate in print before vision loss and so instruction focuses more on developing the tactile and motor skills needed to read braille."

I think the device I am trying to prototype in this project may assist Braille learning for visually impaired children or adults who lost their sight at a later stage of their life.

Video Demonstration

Please check out the following video demonstration:-

Prototype Demonstration

I have pressed : 'H' 'E' 'L' 'L' 'O' 'W' 'O' 'R' 'L' 'D' with Braille keys and listened to the letters audio played back to me

Hardware Build

Following hardware will be used to prototype this solution:

Sony Spresense board: This is an Arduino compatible board with Sony’s CXD5602microcontroller with high-resolution audio capabilities. One of the main reason for choosing this board is it's audio playback capabilities from mp3 files.

I will be using both the Sony Spresense main board and extension board, because the extension board supports I/O expansion, Micro SD, Speaker/Headphone driver + 3.5 mm audio out, I/O logic level shifting etc.

About CXD5602 MCU

The CXD5602GF/GG is a 32-bit RISC microprocessor designed for low-power embedded applications. It features the Arm® Cortex®-M4 processor with a floating-point unit (FPU) and includes the Arm® Cortex®-M0+ processor, which manages system functions such as power management, clock control, and reset, as well as handling I/O processing.

The chip comes with

1.5 MBytes of embedded SRAM
64 KBytes of backup SRAM
256 KBytes of SRAM dedicated to I/O processing

Both the Arm® Cortex®-M4 processor with FPU and the Arm® Cortex®-M0+ are managed by a Power Management Unit, which can power off the processors through internal switches. This allows the SRAM to retain data and enables rapid processor restarts. The Audio Codec Domain supports advanced features such as digital noise cancellation and a digital equalizer, audio recording and playback etc.

4x4 Tactile Button Keypad 1: This is 4 Row x 4 Col = total 16 tactile buttons in a 4x4 matrix patter to take the user inputs for A to P alphabets as follows:

A B C D

E F G H

I J K L

M N O P

All these letters corresponding Braille key caps are placed on each tactile button.

2x5 Tactile Button Keypad 2:This is 2 Row x 5 Col = total 10 tactile buttons in a 2x5 matrix patter to take the user inputs for Q to Z alphabets.

All these letters corresponding Braille key caps are placed on each tactile button.

Q R S T U

V W X Y Z

Both keypad 1 and 2 have total 26 tactile buttons for A to Z all the letters.

Headphone/Speaker: A headphone/earphone can be directly connected to the 3.5 mm Audio output port on the Sony Spresence extension board for listening to the playback sounds.

Alternatively an external speaker can be connected to the Sony Spresence board following this official tutorial:-

Driving Speaker with Sony Spresense board

MicroSD Memory Card: Micro SD card will store the audio files in mp3 format. I have copied 26 mp3 files each playing A, B, C...... X, Y, Z sounds in the memory card.

An audio decoder codec is required in the memory card (details below)
All files to be kept in the root directory of the microSD card
MicroSD needs to be formatted as FAT file system

All these hardware are connected according to the following schematic:

Circuit Diagram

The hardware prototype looks like this after connecting everything

1 / 2

3D Printing or Improvisation ?

If you have access to 3D printer, you may print 3D key caps with Braille dots similar to following design or make your own designs :

3D design for Key caps with Braille Systems

Source : https://grabcad.com/library/braille-key-caps-1

Since, I don't have access to 3D printers, I have improvised a prototyping solution for demonstration purpose.

Please note that, the Braille dots in my design are not exactly following Standard Braille specifications, as this is a handmade design for demonstration purposes.

What I did !

To make somewhat similar to Braille's raised dots, I have basically superglued tiny SMD resistors on key caps ! I wish I had the means to make real Braille dots according to standard specification

Braille on tactile switch key caps

If you want to make actual solutions for this, you need to follow the Braille Specification to make the Braille dots.

source : https://www.compliancesigns.com/blog/what-every-business-should-know-about-braille-signs/

Programming

Step 1: Download Arduino and Add Sony Spersence Board Support

First, download Arduino IDE from following official link:

https://www.arduino.cc/en/software

If you already have any Arduino version 1.8 or later installed then skip this part.

Now, launch Arduino IDE and from File > Preference

Add board support packages for Sony Spresense development:-

Then follow the official guide to setup the board :

https://developer.sony.com/spresense/development-guides/arduino_set_up_en.html

Step 2: Burn Latest Bootloader into CXD5602 for Arduino support

Connect Sony Spresense main board to PC with a micro USB cable
Once the board is detected and setup, update the latest bootloader.

Tools > Board > Spersense

Select, following settings:

Debug: Disables
Core: MainCore
Memory: 768 KB
Upload Speed: 115200

Then click 'Burn Bootloader', this may take a few minutes.

Step 3:Flash mp3 decoder codec into microSD

To be able to play the mp3 audio files with Sony Spresense board, you will need to install mp3 decoder codec in the microSD card

Go to : File > Examples > Audio > dsp_installer > mp3_dec_installer

upload this code into the board for now to install the mp3 decoder.

For more details please check out the official tutorial :

https://developer.sony.com/spresense/development-guides/arduino_tutorials_en.html#_how_to_install_dsp

Step 4: Copy mp3 files to microSD

Extract and Copy all the mp3 files (attached below in zip) to the root directory of the micro SD card.

mp3 files in MicroSD

You may need a card reader or you can use your phone too

Step 5: Flash the sketch (code) into the board

Finally, upload the code I have attached below for this project, after upload the board will reboot and now you can test the prototype.

How to Use it for Learning the Braille ?

Now, I will describe how to use this device to learn Braille

Learning Braille

Step 1: Place index finger of either hand on any keycap
Step 2: Try to sense the raised dots
Step 3: Create a mental image of the number of dots and their positions
Step 4: Press the button and listen to the letter sound
Step 5: Repeat this and memorize the Braille

Once the user feels confident that they can identify Braille, it's time to self practice.

Practicing Braille

Step 1: Place index finger of either hand on any keycap
Step 2: Try to sense the raised dots
Step 3: Create a mental image of the number of dots and their positions
Step 4: Try to guess the letter that is represented by that Braille pattern
Step 5: Press the button and listen to the letter sound
Step 6: Confirm if you have guessed the letter correctly or not
Step 7: Pat yourself in the back and score your performance

Application Areas

Here are some use cases for this solution:

Braille learning access at Libraries/Schools
Braille self learning for sighted people
Braille toy for kids with poor sight/blindness
Memory mapping games
Communication tools for people who are blind and mute
Musical Greeting Cards that plays when Braille Dots are touched

Conclusion

This project may not be that complicated which uses things like AI and IoT but it's a very simple one, yet this can be a nice tool for the visually impaired people. While this prototype is for English letters, it can be easily adapted for other languages.

For low cost affordable solutions, there is not need to use Sony's expensive microcontroller. An ASIC chip that can take multiple touch/press inputs and playback audio should be a great way to make this into a final product. The prototype demonstrated here is a partially developed solution, so there's room for improvements.

I think visually impaired children/people can benefit from this type of solutions.

References

Sony Spresense Official Website

Understanding Braille Systems

Children with Disabilities UNICEF

Build2gether 2.0 Inclusive Innovation

Schematics

Code

FeelTheDotsBraille

#include <SDHCI.h>
#include <Audio.h>
#include <Keypad.h>



const byte ROWS = 4; //four rows
const byte COLS = 4; //four columns

//define the cymbols on the buttons of the keypads
char hexaKeys1[ROWS][COLS] = 
{
  {'A','B','C','D'},
  {'E','F','G','H'},
  {'I','J','K','L'},
  {'M','N','O','P'}
};


const byte ROWS2 = 2; // two rows
const byte COLS2 = 5; //five columns


char hexaKeys2[ROWS2][COLS2] = 
{
  {'Q','R','S','T','U'},
  {'V','W','X','Y','Z'}

};


byte rowPins1[ROWS] = {3, 2, 1, 0}; //connect to the row pinouts of the keypad1
byte colPins1[COLS] = {7, 6, 5, 4}; //connect to the column pinouts of the keypad1

byte rowPins2[ROWS2] = {14, 15};       //connect to the row pinouts of the keypad2
byte colPins2[COLS2] = {8,9,10,11,12}; //connect to the col pinouts of the keypad2


//initialize an instance of class NewKeypad
Keypad customKeypad1 = Keypad( makeKeymap(hexaKeys1), rowPins1, colPins1, ROWS, COLS); 
Keypad customKeypad2 = Keypad( makeKeymap(hexaKeys2), rowPins2, colPins2, ROWS2, COLS2); 


// init SD card with mp3 files and audio playback library

SDClass theSD;
AudioClass *theAudio;

File myFile;
bool ErrEnd = false;
static void audio_attention_cb(const ErrorAttentionParam *atprm)
{  
  if (atprm->error_code >= AS_ATTENTION_CODE_WARNING)
    {
      ErrEnd = true;
   }
}


void setup(){
    while (!theSD.begin())
    {
    }

  // start audio system
  theAudio = AudioClass::getInstance();
  theAudio->begin(audio_attention_cb);
  puts("initialization Audio Library");

  /* Set clock mode to normal */
  theAudio->setRenderingClockMode(AS_CLKMODE_NORMAL);
  theAudio->setPlayerMode(AS_SETPLAYER_OUTPUTDEVICE_SPHP, AS_SP_DRV_MODE_LINEOUT);

}
  

void loop()
{
  err_t err = theAudio->initPlayer(AudioClass::Player0, AS_CODECTYPE_MP3, "/mnt/sd0/BIN", AS_SAMPLINGRATE_AUTO, AS_CHANNEL_STEREO);

  char customKey1 = customKeypad1.getKey();
  char customKey2 = customKeypad2.getKey();
  
  if (customKey1)
  {
  myFile = theSD.open(String(customKey1)+".mp3");
  err = theAudio->writeFrames(AudioClass::Player0, myFile);
  theAudio->startPlayer(AudioClass::Player0);
  theAudio->setVolume(-120);
  theAudio->startPlayer(AudioClass::Player0);
  delay(1000);
  theAudio->stopPlayer(AudioClass::Player0);
  myFile.close();
  }

  if (customKey2)
  {

  myFile = theSD.open(String(customKey2)+".mp3");
  err = theAudio->writeFrames(AudioClass::Player0, myFile);
  theAudio->startPlayer(AudioClass::Player0);
  theAudio->setVolume(-120);
  theAudio->startPlayer(AudioClass::Player0);
  delay(1000);
  theAudio->stopPlayer(AudioClass::Player0);
  myFile.close();
  }
/*
  int error = theAudio->writeFrames(AudioClass::Player0, myFile);

  if (error)
    {
      goto stop_player;
    }

  if (ErrEnd)
    { 
     goto stop_player;
    }


  usleep(40000);


  return;

stop_player:
  theAudio->stopPlayer(AudioClass::Player0);
  myFile.close();
  theAudio->setReadyMode();
  theAudio->end();
  exit(1);
*/
}

Credits

Shahariar

74 projects • 266 followers

"What Kills a 'Great life' is a 'Good Life', which is Living a Life Inside While Loop"

Comments

Awards

Lucky Draw for TWO Submissions

Build2gether 2.0 — Inclusive Innovation Challenge

Braille Trainer Board: Learning Dots with Auditory Feedback