Payment Terminal for the Visually Challenged

Story

What is a payment terminal?

A point of sale terminal (POS terminal) is an electronic device used to process card payments at retail locations.Nowadays payment terminals are used almost everywhere to accept digital payments / transactions in local stores, coffee shops, hospitals, hotels, etc.

What is the observed problem with general POS systems?

Generally it is observed that "no" payment terminals are specially dedicated for visually challenged people. The conventional payment terminals consists of touchscreens / LCDs and has no user interface designed for visually challenged People. If visually challenged people prefer card payment, they need to take help of others while making payment through card & has security related issues for them involved in the transaction.

Overview

The payment process would be as depicted in the flowchart:

1. The headphones would be first put on by the user. LDR detects whether the headphones are properly put on. It ensures the saftey while the transaction is processed. The transaction gets cancelled if the headphones are removed.

2. The card is swiped/ read by the RFID reader/magnetic strip sensor and checks whether the card is valid or not. If it is not valid then it stops the program.

3. The input is taken from the numpad keys attached to the Sony Spresense board.

4. The appropriate audio (.mp3) files are played that are stored onto the SD card.

The basic input and output blocks to the Spresense controller are as shown below

Setting Up the Audio

In order to play the.mp3 files from the SD card we need to first install the DSP codec files. I prefer to install them onto the SPI-Flash as follows:

The DSP installer is launched from Arduino IDE, to open it click on the toolbar menu: File → Examples → Examples for Spresense Audio → dsp_installer. Press 1 to install onto SPI-Flash or press 2 to install onto SD card.

Note: 1. Use a FAT-32 formatted SD card; 2. SD card capacity should not be more than 32GB; 3. The audio application calls the initPlayer() or initRecorder() function with the argument of the DSP installation directory. If you installed DSP binary to the SD card, you have to specify the /mnt/sd0/BIN. If you installed into the SPI Flash, you have to specify the /mnt/spif/BIN.

To make the.mp3 files I used the Balabolka software that converts text into natrual language speech using Microsoft Api. The files were saved onto the SD card. I created the play_sound() function for ease of use of playing audio files later in the main sketch.

#include <SDHCI.h>
#include <Audio.h>
SDClass theSD;
AudioClass *theAudio;
File myFile;
bool ErrEnd = false;
/*Delcaration of audio_attention_cb funtion */
static void audio_attention_cb(const ErrorAttentionParam *atprm){
   puts("Attention!");
   if (atprm->error_code >= AS_ATTENTION_CODE_WARNING){
       ErrEnd = true;
   }
}
void setup(){
}
void loop()
{
   play_audio("Sound.mp3",40000);
}
void play_audio(String filename,int delay_time) 
{  
// Generally delay time is 40000
   /* Setup the audio player */
   theAudio = AudioClass::getInstance();
   theAudio->begin(audio_attention_cb);
   puts("initialization Audio Library");
   theAudio->setRenderingClockMode(AS_CLKMODE_NORMAL);
   theAudio->setPlayerMode(AS_SETPLAYER_OUTPUTDEVICE_SPHP, AS_SP_DRV_MODE_LINEOUT);
   err_t err = theAudio->initPlayer(AudioClass::Player0, AS_CODECTYPE_MP3, "/mnt/spif/BIN", AS_SAMPLINGRATE_AUTO, AS_CHANNEL_STEREO);
   if (err != AUDIOLIB_ECODE_OK){
       printf("Player0 initialize error\n");
       exit(1);
   }
   myFile = theSD.open(filename); // Takes the file name as the function parameter
   if (!myFile){
       printf("File open error\n");
       exit(1);
   }
   printf("Open! %d\n",myFile);
   err = theAudio->writeFrames(AudioClass::Player0, myFile);
   if ((err != AUDIOLIB_ECODE_OK) && (err != AUDIOLIB_ECODE_FILEEND)){
       printf("File Read Error! =%d\n",err);
       myFile.close();
       exit(1);
   }
   puts("Play!");
   theAudio->setVolume(-160);
   theAudio->startPlayer(AudioClass::Player0);
   while(1){
       puts("loop!!");
       int err = theAudio->writeFrames(AudioClass::Player0, myFile);
       if (err == AUDIOLIB_ECODE_FILEEND){
           printf("Main player File End!\n");
           break;
       }
       if (err){
           printf("Main player error code: %d\n", err);
              sleep(1);
           theAudio->stopPlayer(AudioClass::Player0);
           myFile.close();
           break; // Break the while loop
           exit(1);  
       }
       if (ErrEnd){
           printf("Error End\n");
              sleep(1);
           theAudio->stopPlayer(AudioClass::Player0);
           myFile.close();
           break; // Break the while loop
           exit(1);  
       }
       usleep(40000); 
   }
}

However when I played these text to speech files I observed that the sampling rate detected by the spresense board was too high which played the files very fast. I tried tweaking the follwing line of code in the above program, err_t err = theAudio->initPlayer(AudioClass::Player0, AS_CODECTYPE_MP3, "/mnt/spif/BIN", AS_SAMPLINGRATE_AUTO, AS_CHANNEL_STEREO); as I thought changing the AS_SAMPLINGRATE_AUTO will correct it. I tried putting 32000 or AS_SAMPLINGRATE_32000 in the place of AS_SAMPLINGRATE_AUTO but it didnt work.

I did find a solution to the above issue however it is not the best solution, but it did get my project working. What I did was to merge the text to speech file with a music file. The spresense controller adjusts to the sampling rate of the music file and hence the text to speech files are played properly. I used https://mergeaudio.online/ to merge the.mp3 files.

Setting Up the Keypad

The keypad/numpad is a 4*3 keypad and its purpose is to take the input from the user for entering the pin and chosing the options to continue or go back.

I created the numpad() function for ease of use of verifying the pin entered by the user later in the main sketch. The below program can be used to test the numpad separetly.

/* This program is used to check the working of the fucntion check_pin */
/* The check_pin function returns a boolean value and does not take any parameters*/
#include<Keypad.h>  // Aurduino Library for keypad
#define Password_Length 5  // Define the password length + the null character
char Data[Password_Length]; // Declare the array to store the pin
char Master[Password_Length] = "1234";  // Declare the password *Here it is declared for prototype purposes(Can use API) 
byte data_count = 0, master_count = 0;  
char customKey; // To store the typed character on the keypad
const byte ROWS = 4;  // Declare the number of rows for the keypad
const byte COLS = 4;  // Declare the number of columns for the keypad
// Here I have used a 4 * 3 keypad hence there are 7 keys
char hexaKeys[ROWS][COLS] = {
 {'1', '2', '3'},
 {'4', '5', '6'},
 {'7', '8', '9'},
 {'*', '0', '#'}
};
byte rowPins[ROWS] = {8, 7, 6,5};  // Delcare the pins to which the corresponding row is attached on the Spresense board
byte colPins[COLS] = { 4, 3, 2};   // Declare the pins to which the corresponding column is attached on the Spresense board
Keypad customKeypad = Keypad(makeKeymap(hexaKeys), rowPins, colPins, ROWS, COLS); // Create instance of the class keypad from the Keypad.h library
void setup(){
 Serial.begin(115200);  // Declare the baud rate , Here I am using 115200
 pinMode(LED0,OUTPUT);  // LED0 represents the led on the Spresense board
}
void loop(){
 check_pin();
}
void clearData(){
 while(data_count !=0){
   Data[data_count--] = 0; 
 }
 return;
}
void blink_led0(int choice,int time_delay){
 while(1)
 {
   digitalWrite(LED0,HIGH);
   delay(time_delay);
   digitalWrite(LED0,LOW);
   delay(time_delay);
   if(choice=0)
   {
     break;
   }
 }
 return;
}
bool check_pin(){
 bool pass_check =false; // False if the password is incorrect
 while(1){
   //Serial.print("Enter Password:");  //Uncomment the line for testing 
   customKey = customKeypad.getKey();  // the getKey() method returns the character pressed on the keypad
   if (customKey){
     Data[data_count] = customKey;  
     data_count++; 
   }
   if(data_count == Password_Length-1){
     /* If the number of digits entered matches the number of digits in the set password */
     if(!strcmp(Data, Master)){
       /* If the entered password matches the set password */
       //Serial.print("Correct"); // Uncomment the line for testing
       clearData(); // Clears the data in the array that holds the user typed pin
       /* If the password matches then enter this loop */
       //Serial.println("Password Matching"); // Uncomment the line for testing
       pass_check = true;
       break;
     }
     else{
     //Serial.print("Incorrect");  // Uncomment the line for testing
     delay(1000); // This delay is used to ensure proper detection of the keys pressed
     }
   }
 }
 if(pass_check){
   return true;
 }
 else{
   return false;
 }
}

Setting up Headphones

I modified the headphones and attached a LDR( Light Dependent Resistor) inside it.

The purpose of the LDR is to detect whether the headphones are put on by the user or not as this would lead the the saftey of the information being exchanged between the user and device. The resistance of the LDR will vary when the heaphones are removed or when they are put on because in that case light will be blocked by the user.

The following program can be used to test the functioning of the headphones along with the LDR. I created the ldr_detect() function for ease of of verifying whether the headphones are put on the user, it is used later in the main sketch.

/* This program is used to test the ldr_detect function */
#define ldr1 A0  // ldr on the inner left side of the headphone
void setup()
{
   Serial.begin(115200); // Here the baud rate is set to 115200. It can be varied.
   pinMode(ldr1,INPUT); // Declares ldr pins at input
}
void loop()
{
  bool check_headphone = ldr_detect(300); // Check headphone stores the value whether headphones are put on or not
  if(check_headphone)
  {
      //Serial.println("Headphones are put on hence the function is working"); // Uncomment this line for testing
  }
  else
  {
      //Serial.pirntln("Headphones are not put on but the function is working"); // Uncomment this line for testing
  }
}
bool ldr_detect(int light_threshold){  
   //Pass the value of light_threshold which it should detect that the headphones are put on (0-1023). Generally the value is 500
   /* This function is used to detect whether the headphone is put on or not */         
   int ldr1_val = analogRead(ldr1);
   if(ldr1_val<=light_threshold)
   {
       //Serial.println("The headphones are put on by the user"); // Uncomment this line for testing
       return true;
   }
   else
   {
       //Serial.println("The headphones are not put on by the user"); // Uncomment this line for testing
       return false;
   }
}

Setting Up the RFID

The RFID sensor used in this project for prototype is the MFRC522 sensor and RFID card. However the c++11 used in MFRC522 is not supported by spresense yet. Hence it can be directly wired up with the spresense board when the support is brought. So I decided to use an Arduino Uno board to verify the card of the user and then send a signal to the Spresense board that the card is verified. ***This is done for the sole reason of prototyping. On further development the verification can be done using an API.

The below code is added to the main sketch that receives the signal from the Uno board that the card is verified.

/***********CHECKS WHETHER VALID CARD IS DETECTED WITH ARDUINO*********************/
       /*******The below code part can be replaced with actual code of rfid after sepresense supports c++11**/
       while(1){
           //Serial.println("Check whether card is valid");  // Uncomment this liner testing
           //Serial.println(digitalRead(12));  // Uncomment this line for testing
           if(digitalRead(12)){
               break;
           } 
       }
       //Serial.println("Check card program ended"); // Uncomment this line for testing
       /***************************************************************************/

Also the Uno is codded to turn its pin HIGH if the RFID is verified. Since the below code is solely for prototyping I have not added it to the main sketch. The below code is uploaded in the Arduino Uno.

#include <SPI.h>
#include <MFRC522.h>
#define SS_PIN 10   // SDA is connected to pin 10 of arduino
#define RST_PIN 9   // RESET is connected to pin 9 of arduino
MFRC522 mfrc522(SS_PIN, RST_PIN);   // Create MFRC522 instance.
String card_number = "91 47 BE 2F";    // Enter the card number for prototype
void setup() 
{
   Serial.begin(115200);   // Initiate a serial communication
   SPI.begin();      // Initiate  SPI bus
   mfrc522.PCD_Init();   // Initiate MFRC522
   Serial.println("Approximate your card to the reader...");   //Uncomment this line for testing
   pinMode(7,OUTPUT); // Send the signal that rfid is verified
}
void loop() 
{       
       digitalWrite(7,LOW);
       bool verified = check_card(card_number);
       if(verified){
          while(1){
           Serial.println("The card is verified");
           digitalWrite(7,HIGH);
          }
       }
       else{
           Serial.println("The card is not verified");
       }
} 
bool check_card(String card_number)
{
   int val = 0;    // Stores whether the card is verified or not
   while(1){
       if ( ! mfrc522.PICC_IsNewCardPresent()) {
           // Look for new cards
           continue;
       }
       if ( ! mfrc522.PICC_ReadCardSerial()) {
           // Select one of the cards
           continue;
       }
       //Show UID on serial monitor
       //Serial.print("UID tag :");    //Uncomment this line for testing
       String content= "";
       byte letter;
       for (byte i = 0; i < mfrc522.uid.size; i++) 
       {
           //Serial.print(mfrc522.uid.uidByte[i] < 0x10 ? " 0" : " "); // Uncomment this line for testing
           //Serial.print(mfrc522.uid.uidByte[i], HEX);    //Uncomment this line for testing
           content.concat(String(mfrc522.uid.uidByte[i] < 0x10 ? " 0" : " "));
           content.concat(String(mfrc522.uid.uidByte[i], HEX));
       }
       //Serial.print("Message : ");   // Uncomment this line for testing   
       content.toUpperCase();
       if (content.substring(1) == card_number){
           //Change the UID to give access or use api to fetch it here
           Serial.println("Authorized access");  // Uncomment this line for testing
           delay(3000);
           val = 1;
           break;
       }
       else{
           Serial.println(" Access denied"); // Uncomment this line for testing
           delay(3000);
           val = 0;
           break;
       }
   }
   if(val){
       return true;
   }
   else{
       return false;
   }
}

Note : Pin 7 of the arduino uno is connected to pin 12 of the Spresense board

Comments

Since Standard Payment Termainals focus on great displays and touch-screen type of User Interfaces which are of no use to Visually Challenged.This system has a unique audio and sensory User Interface which is of great use for Visually challenged and hence improving their daily lives and experience in shopping malls, booking train tickets, booking bus tickets, payments at hotels etc. And also profiting businesses by imporving their customer satisfaction.Also regional language features can be easily added since.mp3 files are played.

Prototype Demonstration

In this demonstration, I am showing the overall working of the prototype, the video is taken by me and the prototype is demonstrated by my friend. The user would hear the audio in his/her headphones but for the purpose of demonstration I have connected it to a speaker. Hope you enjoy the video!