Drishya: Smart glass aiding the visually impaired

Idea behind the project

Mr Prakash Timilsina, from Nepal, is an internationally renowned singer. He wants to perform his daily activities like moving indoors and outdoors, reading his musical sheets, walking safely in the road and going to different places independently. But he cannot do so because he is visually impaired by birth. He always needs his wife or a friend to assist him in his daily activities. He wonders if he has any answer to his problems that can help him to live an ordinary life like others. Although there are products available in the market, like the Smart blind canes that can help him to walk, those products cannot address all his problems and needs. Accessibility is an afterthought, be it an improvement in society, culture, or technology. Though we have billions of dollars invested in the field of Metaverse, for people who can see; what about the people who cannot see? About 90% of the world's visually impaired people are living in developing countries. They cannot easily recognize obstacles, people, travel routes independently, or use stairs properly. They struggle with their day-to-day life and everyday activities. No safety features are present on the normal blind stick which is mostly used by visually impaired people as for their aid and support.

Our solution is to develop a smart glass which will not only be able to predict the objects but also find their location and provide feedback to the user. The visually impaired person will be provided with the feedback through speakers that would be present inside the arm of the smart glasses. In our prototype, the bounding boxes in the FOV of the camera will be recited to the ears of the user with a simple location-based quadrant mapping with a central field. If an object is in any of these locations, its position along with the object type will be played through the speakers. This enables the user to be conscious of their surroundings.

We will be connecting our smart glass with a protocol-like matter which will be able to integrate with all the smart home automation devices and other security devices that are present throughout the house making the life of the visually impaired much easier. Also, different existing solutions and services developed for people who can see will be integrated making our users feel welcomed. Main examples include map services to help them navigate, traffic signal integrations to help them cross the road, the status of different home appliances, and so on. This solution makes the day-to-day activity of the visually impaired people much easier. Our solution tends to provide a means to ease the life of a visually impaired person.

Feedback

We worked further on our prototype and got it tested with Mr Prakash for his feedback as he will be one of our target users in the near future. He was really excited when we explained to him the current and upcoming features of our device. He was even happier when he got hands-on experience of our prototype. Mr Prakash is also well-known and has experienced technologies like Android Assistive and Google Glass. He really enjoyed our prototype and provided us with feedback on the features that we should integrate like Navigation, Person Recognition, Calling, and creating a compact device, which we had already planned. We also are developing a 3D model of our device where we can integrate the entire system in these glasses. The main reason for getting our prototype tested and discussed with Mr Prakash, before even developing an MVP is, we wanted to include features, provide benefits and make our product suitable for a visually impaired person. If we didn’t do the testing, we would not have known the requirements of our user. So, rather than thinking and developing the solution with only addressing the problems that we think exists, we got it tested to know about the problem that our users actually face.