Vocable AAC: Conversational AI Meets Generative AI

It Began With One Of Our Own.

When former WillowTree designer Matt Kubota’s partner Ana was diagnosed with Guillain-Barré Syndrome, we knew we had to help.

The rare autoimmune disorder left Ana temporarily paralyzed and unable to speak. Ana and her caregivers looked to augmentative and alternative communication (AAC) devices. But they soon discovered the available options were expensive, bulky, rudimentary, or outdated. While Ana recovered, she painstakingly communicated by blinking as caregivers pointed to letters on an alphabet poster.

Matt knew he and the WillowTree team could create a better experience for Ana and other AAC users: stroke and trauma survivors, people with MS or ALS, non-verbal individuals with autism, intubated hospital patients, and many more. That vision became the Vocable AAC app, a revolutionary augmentative and alternative communication tool, now AI-enabled with the integration of ChatGPT.

Digital Strategy Consulting

Leveraging Research to Reduce Call Volume

To understand pain points in Ascensus' existing digital touchpoints, WillowTree’s Digital Strategy team dove into qualitative research, including:

• Facilitating our Outcomes Workshops with Ascensus stakeholders to align on North Star outcomes
• Interviewing Ascensus call center representatives to identify common issues customers faced during registration and onboarding
• Conducting user interviews with a diverse cross-section of employers and HR/payroll professionals to unpack their challenges

We Believe in the Power of Voice

We believe shared language lies at the root of every relationship. Language connects every parent and child, every caregiver and patient, every programmer and piece of technology, creating something greater than ourselves: community.

Vocable is making a major difference in the ALS community. I have seen people break into tears when using your app.Tom MeadowsFounder, Steve’s Way

Tracking Eye Movements

Paralyzed users like Ana, who can’t tap a screen, needed an innovative way to manipulate an AAC device, and the power of the WillowTree community proved itself to Matt and the Vocable development team repeatedly. In the product’s first iteration — “Eye Lasers” — users experienced significant shakiness when trying to control the cursors that tracked their eye and head movements. ‍ The robotics expertise of WillowTree senior software engineer Chris Stroud unlocked the solution, providing a more stable, precise experience when manipulating elements of any size and on any device. With subtle head movements, users could now control a cursor to type letters or select custom phrases from personalized categories.

Personalized Pattern Detection

The integration of OpenAI's ChatGPT allows the app to retain conversational context, enhance predictive pattern detection, and bolster semantic understanding of a caregiver’s speech, offering a faster, smoother, and more natural conversational experience for today’s AAC users. It's an ideal use case for conversational + generative AI.

Activating Listening Mode

Vocable is a good listener, too. Through Listening Mode, Vocable understands caregiver speech and context, offering users AI-generated responses that carry the conversation forward. Users or caregivers can quickly and easily activate Listening Mode by tapping the mic icon or saying, “Hey, Vocable.”

ChatGPT is a conversational interface, it's literally made for this. It understands an entire conversation's history, providing better AI-generated responses for the non-verbal user to select.Andrew CarterWillowTree Principal Practice Advisor

UX and Accessibility Considerations

If users have to spend so much of their day interacting with the Vocable app, it’s gotta look good, too, right? Based on feedback from research partners like the assistive technology teams at Duke University and speech-language pathologists, WillowTree’s design team identified ways to accomplish exactly that. Guiding our design work was the principle that high contrast can be so much more than black and white. That’s why Vocable incorporates the strongest ADA compliance through elements like a compelling high-contrast palette and large typography, promising a consistent and aesthetically pleasing experience across devices. Because what’s accessible should also be enjoyable.

When so few hospitals have available technology for the communication-vulnerable, it's exciting that a platform like Vocable exists. Vocable is easy to access and most importantly free.Dr. John M. CostelloMA, Boston Children’s Hospital