How Thought-Controlled Wheelchairs Work

Ambient Audeo System

Scientist Chuck Jorgensen uses a subvocal speech-recognition device to move a simulated Mars rover.
Scientist Chuck Jorgensen uses a subvocal speech-recognition device to move a simulated Mars rover.
NASA Ames Research Center, Dominic Hart

Michael Callahan and Thomas Coleman founded Ambient, the company that develops and markets the Audeo system. Audeo was initially envisioned as a way for severely disabled people to communicate, but Ambient expanded the control systems to include the ability to control a wheelchair or interact with a computer.

The Audeo is based on the idea that neurological signals sent from the brain to the throat area to initiate speech still get there even if the spinal cord is damaged or the motor neurons and muscles in the throat no longer work properly. Thus, even if you can't form understandable words, neurological signals that represent the intended speech exist. This is known as subvocal speech. Everyone performs subvocal speech -- if you think a word or sentence without saying it out loud, your brain still sends the signals to your mouth and throat.

A lightweight receiver on the subject's neck (a small array of sensors attached near the Adam's apple area) intercepts these signals. It functions much like an electroencephalogram, a device that can receive neurological signals when placed on a subject's scalp. The Audeo receives specific speech-related signals because it is placed directly on the neck and throat area. The sensors in the receiver detect the tiny electric potentials that represent neurological activity. It then encrypts those signals before sending them wirelessly to a computer. The computer processes the signals and interprets what the user intended to say or do. The computer then sends command signals to the wheelchair or to a voice processor.

Here is an example of the Audeo system in action: You want to say, "Hello, how are you?" and say it silently in your mind. Your brain sends signals to the motor neurons in your mouth and throat. The signals are the same as the ones that would be sent if you had really said it out loud. The Audeo receiver placed on your throat registers the signals and sends them to the computer. The computer knows the signals for different words and phonemes (small units of spoken speech), so it interprets the signals and processes them into a sentence. It works in much the same way as voice-recognition software. The computer finishes the process by sending an electronic signal to a set of speakers. The speakers then "say" the phrase.

If you want to control a wheelchair, the process is similar, except you learn certain subvocal phrases that the computer interprets as control commands rather than spoken words. The user thinks, "forward," and the Audeo processes that signal as a command to move the wheelchair forward.

Audeo uses a National Instruments CompactRIO controller to collect the data coming from the sensors. Embedded software known as LabVIEW then crunches the numbers and converts the signals into control functions, such as synthesized words or wheelchair controls. Ambient has developed the communication aspect of Audeo to the point that users can create continuous speech, rather than speaking on word at a time [source: Ambient].