Be My Eyes Ties OpenAI's GPT-4 Into Its Accessibility Service — and Aims to Improve Blind Users' CX

Assistive service Be My Eyes, which links blind and low-vision users with seeing volunteers and staff, has announced a partnership with Microsoft to bring artificial intelligence (AI) image query support to its customer service platform — claiming "stunning results" from what it's calling Be My AI.

"Our new AI-powered accessible customer service model is producing remarkable results for not only one of the world's largest and most innovative companies, but also for a community of blind and low vision consumers," claims Mike Buckley, Be My Eyes' chief executive officer.

"Be My AI is the customer service gold standard for companies that want to assist blind and low vision consumers, and we are incredibly thankful to Microsoft for their leadership, willingness to partner on Accessible CX [Custom Experience] solutions and their help communicating the benefits of Be My AI to other enterprises so we can advance our mission to make the world more accessible."

Launched in 2015, Be My Eyes connects volunteers and staff with blind and low-vision users via a smartphone app. Video is streamed from the phone, and the volunteer or staff member describes what they can see and answers any questions that crop up. For its users, the service is invaluable — but it also struggles to scale, requiring one volunteer or staff member per active user at any given time.

That's where Be My AI comes in. Developed based on OpenAI's GPT-4 vision model, the artificial intelligence system analyses visuals and provides text-based responses — not replacing the volunteers and staff but complementing them, while also making the service more available to deaf-blind users by supporting Braille display systems. Should the AI fail to answer a query, the user can then be routed through to a human as with the original implementation.

"The large and growing number of blind and low vision people globally, roughly equivalent to the population of the United States, means every company, non-profit, academic institution, and public sector organization has both a moral and business imperative to better serve our community," says Mark Riccobono, President of the National Federation of the Blind.

"The deep commitment of Be My Eyes to have their product development driven by the lived experience of blind people is truly transformative," Riccobono continues. "As a result, Be My AI is one of the most exciting technological assistance solutions we have ever evaluated, and given the initial results in the customer service environment, which has historically been a significant source of frustration for blind consumers, we encourage every organization to evaluate this remarkable tool as an important option."

Having proven the concept in-app, the company is now working to move the AI-based technology into third-party customer service platforms — and claims impressive results from its deployment with Microsoft, based on its "3S Success Criteria" metrics: a 90 per cent successful resolution rate; a speed which solves queries in four minutes, on average, compared with 12 for a human agent; and high satisfaction ratings, with the implementation in Microsoft's Disability Answer Desk rated 4.85 out of five stars by its users.

The Be My AI service is now live on the Be My Eyes app and the Microsoft Disability Answer Desk, for answering Microsoft-specific queries; Be My Eyes has confirmed it is interested in working with other companies and organizations to deploy the technology further.