Bring Your Own Accessibility

Touchscreens have become an indispensable part of our modern daily lives, being seamlessly integrated into various public and private environments such as kiosks, ATMs, smartphones, and even home appliances. Their intuitive interface and interactive capabilities have transformed the way we interact with technology, providing a user-friendly and engaging experience for individuals across all age groups. These devices have streamlined tasks and transactions, allowing users to effortlessly navigate complex processes with the simple touch of a finger.

However, despite their widespread adoption, the lack of accessibility of touchscreen technology remains a subject of concern. For individuals with certain physical disabilities, such as poor vision, tremors, or muscle spasms, these interfaces can present significant challenges, rendering them essentially unusable. The standard gestures and visual-based interactions that come naturally to many users can pose insurmountable obstacles for those with disabilities, leading to frustration and exclusion from essential services and technological advancements.

Efforts have been made to address these accessibility concerns, leading to the development of specialized hardware and software solutions tailored to accommodate diverse user needs. However, the need for specialized hardware and the necessity for users to learn a new set of gestures or interaction patterns has hindered the widespread adoption of accessible touchscreen interfaces. Moreover, the costs associated with these specialized solutions have posed additional barriers, further impeding their integration into mainstream technology. As a result, a considerable segment of the population continues to face challenges in accessing and utilizing touchscreen devices, perpetuating the digital divide and hindering their full participation in the digital age.

A completely different approach taken by engineers at the University of Michigan may soon allow individuals with physical disabilities to interact with any touchscreen. They developed a smartphone case named BrushLens that works with a smartphone to effectively convert any standard touchscreen interface into an accessible interface. BrushLens works to help users first understand the options available on the display, then pivots to assist them in locating and physically pressing the desired button.

BrushLens takes the form of a smartphone case. After snapping their phone into the case, a user can slide it across the inaccessible touchscreen. While doing so, the system will use the phone’s camera and processing power to run an app that audibly speaks the name of each interactive element on the screen. These options are then presented as a list of oversized buttons on the smartphone screen that the user can select.

Once a selection has been made, the BrushLens system will switch into its localization and actuation mode. It will continually announce the phone’s present position, as well as the target position that the phone needs to be moved to. The position of the phone is determined using a combination of images captured from its camera and measurements from its inertial measurement unit.

The moment that the position of the phone overlaps with the selected target on the screen, BrushLens will switch into its actuation mode. The team demonstrated two options for how this mode can work. One option leverages a number of solenoids covered with a conductive fabric that can extend to physically touch the screen, much like a fingertip. This option offers compatibility with virtually any type of touch interface, but is a bit coarse in terms of accuracy. The autoclicker method, on the other hand, changes the capacitance of the touchscreen to trigger a touch action with a high level of precision, but only works with certain types of displays.

A small study consisting of 10 participants was conducted to assess the effectiveness of this new tool. Those with tremors or spasms saw accuracy increases of about 74%, and the visually impaired participants noted that they believed they would be capable of using kiosks that would previously have been impossible to use.

At this time, the team is working towards refining their device for commercialization. They believe it can be produced for under $50.