PTA: Personal Travelling Assistant

What is PTA

This project is built to learn and use IoT in our daily life. PTA is a state of the art virtual travelling partner/assistant that helps you stay connected to not only the real world but also to the digital world as well as your vehicle. It accepts your speech commands and does certain actions for you. It also speaks to you certain important notifications thus helping you stay alert.

Personal Travelling Assistant (PTA) is a gadget that let your vehicle skid into the world of IoT and acts as a gateway to the world of modern vehicles. PTA will let you speak to your car and make it do useful things for you. From entertainment to basic support and from engagement to doing tasks, PTA is one-roof solution and is there to replace your travelling partner.

Why build PTA?

I build this project because most of the times I am travelling on long routes and get bored of driving alone. So, PTA doesnt give me any feeling that I am single. It also helps me keep an eye on the road by doing things herself ensuring me on-road safety. She also keeps on talking to me on different topics which she either chooses from internet (breaking news, hotspots on my route) or talks to me with status of my vehicle.

Basic Structure of PTA

PTA consists of a mobile application built on amateur artificial intelligence (amateur in the sense that it is not competing with other similar available products but still has the basic functionality like them) and a hardware interfacing with the vehicle. Mobile application (Zeeshata) is voice interactive module that listens and speaks to you in a real-time environment. It interprets what you are speaking and based on that it executes certain actions.

Currently the application has been developed for Android phone but soon to be shifted to iOS as well. The hardware consists of an Arduino UNO which acts as supporting processor for other auxiliary devices performing other actions. At the moment, the auxiliary functions consists of approaching vehicle alert, parking assistance and vehicle functional monitoring.

Basic fucnitons of PTA

PTA uses Google's voice recognition service and Text-to-Speech service for 2-way interaction. Digital info is gathered through navigation and internet. Vehicle communication is done with an OBD2 reader and communicates through bluetooth. Other features require certain sensors and their physical communication with PTA.

Mobile App

Before we start discussing this app, I would like to inform that this project has been owned (during its development) by one of the local automation industries and due to this, I will not be able to disclose the source code. But the good news is that I can share the block diagram as how the app is processing the commands and also I can upload the basic app to give a taste of the idea behind this project. This app has been created using Android Eclipse IDE in Java language. Source code guidelines can be provided who want to replicate it.

As can be seen in the block diagram, voice commands are received and based on each command a certain action is done. Commands are recognised through Google voice recognition service and speech actions are done through Text to Speech engine. For example, the user speaks 'Where am I' then the app speaks his current GPS location. This location is not in the form if GPS cordinates but it contains information like Area name, City and Country. If the user wishes to make a phone call then he needs to say 'Make phone call'. Zeeshata (your assistant's name) asks you 'Whom you want to call'. At this stage, the user must take the name of the person which is then searched in the phone directory. Upon successful match, the default number is returned and the phone makes a call using that number. When an SMS is received, Zeeshata asks 'You have received an SMS from (maps the number with name in directory and returns). Do you want me to read for you'. If the user says 'yes' then Zeeshata will speak the complete SMS. Below is screenshot of the app

i shown in the app is to help the user know about valid commands in case he forgets.

Vehicle Interfacing

Zeeshata needs to communicate with vehicle's central computer which is working on CAN protocal. In order to do that, you need to have a special scanner which is connected to OBD2 port of the vehicle. I used below OBD2 scanner/reader for this purpose.

This is device communicates with Zeeshata over bluetooth interface. Zeeshata sends commands over a dedicated threa defined in the app and the broadcast receiver receives the response from OBD2 scanner and the takes pre-defined actions like over-speed alert, engine faults, sensors check and other basic functions under the scope of this project.

Auxiliary Functions

One of the auxiliary functions is already explained above (vehicle functional monitoring). Another fucntion is to alert approaching vehicles from any side. This is achieved through Ultra sonic distance sensor

This sensor works on ultrasonic and uses Sonar principle. It is installed at four ends of the car (Left, RIght, Front and Back) and keeps on calculating the distance of other vehicles from other car. If any sensor detects a vehicle within its alert area (defined in arduino code), it sends alert to Arduino which in turn sends alert to Zeeshata to speak to driver and tells in which direction the vehicle is approaching. There is another alert mechanism which is comprised of small LED installed on both side view mirrors. Corresponding LED starts flashing to alert driver to use that specific mirror to look at approaching vehicle.

Arduino Function

Arduino UNO acts as vehicle's central processor where the code consists of below functions:

• Sonar to read the ultrasonic sensors and do the distance measurements

• Send alert message to Zeeshata through bluetooth

• Flash corresponding LED to alert driver about the approaching vehicle

Arduino Sonar is implemented through 'Ping' and does the distance calculation. LEDs are connected to digital pins of Arduino while bluetooth interface is achieved through HC06 module

This bluetooth module is connected Tx and Rx of UNO and is paired with mobile phone. App has bluetooth broadcast receiver which is always listening to bluetooth paired devices and is communicating with both OBD2 scanner and HC05.

Ending Notes

I really appologies for not uploading the actual code for arduino and mobile app due to the company's ownership. But the block diagram of app interaction as well as the basic app itself will help you rebuild the project. You may contact me for further support and also please do share if you make any enhancements to this project.