School of Electrical and Computer Engineering
Text Box: Tongue Drive System: A Brain-Tongue-Computer Interface Text Box: IV. Tongue Drive System Tongue Drive System (TDS) tracks the tongue motion by an array of magnetic sensors, which measure the magnetic field generated by a small permanent magnet, the size of a grain of rice, that is coated by a biocompatible material such as titanium and attached to the tongue via piercing, implantation, or adhesion. Magnetic sensors can be either mounted on a dental retainer and clipped on the teeth (internal Tongue Drive System or iTDS) or on a headset (external Tongue Drive System or eTDS) positioned near the cheeks. Sensor outputs are amplified, multiplexed, digitized, and transmitted wirelessly to an external controller unit. Signals received by the external controller, which can be a portable computer or a smartphone (e.g. iPhone) are processed to indicate the motion of the permanent magnet and consequently the tongue position within the oral cavity. We can assign a certain control function to each particular tongue movement in the software and customize the system for each individual user. These user-defined control functions may then be used to operate a variety of devices and equipments including computers, phones, and powered wheelchairs. The signals from the magnetic sensors are linear functions of the magnetic field, which is a continuous position-dependent property. Thus a few sensors are able to capture a wide variety of tongue movements. This mechanism can provide a tremendous advantage over switch based devices in that the user will have the options of proportional, fuzzy, or adaptive control, which can offer smoother, faster, and more natural control over the environment.

© 2006 Maysam Ghovanloo

Internal Tongue Drive System (iTDS) consists of a permanent magnet attached to the tongue plus an array of magnetic sensors around the lower teeth on a dental retainer. 

Text Box: Related Publications: J. Kim, X. Huo, J. Minocha, J. Holbrook, A. Laumann, and M. Ghovanloo, “Evaluation of a smartphone platform as a wireless interface between Tongue Drive System and electric-powered wheelchairs,” IEEE Trans. Biomed. Eng., vol. 59, no. 6, pp. 1787–1796, June. 2012. X. Huo, and M. Ghovanloo, “Using speech recognition to enhance the Tongue Drive System functionality in computer access,” Proc. IEEE 33rd Eng. in Med. and Biol. Conf., pp. 6393-6396, Sep. 2011. B. Yousefi, X. Huo, and M. Ghovanloo, “Preliminary assessment of Tongue Drive System in medium term usage for computer access and wheelchair control,” Proc. IEEE 33rd Eng. in Med. and Biol. Conf., pp. 5766-5769, Sep. 2011. E.B. Sadeghian, X. Huo, and M. Ghovanloo, “Command detection and classification in tongue drive assistive technology,” Proc. IEEE 33rd Eng. in Med. and Biol. Conf., pp. 5465-5468, Sep. 2011. H. Park, J. Kim, X. Huo, I. Hwang, and M. Ghovanloo, “New ergonomic headset for Tongue-Drive System with wireless smartphone interface,” Proc. IEEE 33rd Eng. in Med. and Biol. Conf., pp. 7344-7367, Sep. 2011. B. Yousefi, X. Huo, E. Veledar, and M. Ghovanloo, “Quantitative and comparative assessment of learning in a tongue-operated computer input device,” IEEE Trans. Info. Tech. in Biomedicine, vol. 15, no. 5, pp. 747-757, Sep. 2011. B. Yousefi, X. Huo, and M. Ghovanloo, “Using Fitts’s law for evaluating tongue drive system as a pointing device for computer access,” Proc. IEEE 32nd Eng. in Med. and Biol. Conf., pp. 4403-4406, Sep. 2010. J. Kim, X. Huo, and M. Ghovanloo, “Wireless control of smartphones with tongue motion using tongue drive assistive technology,” Proc. IEEE 32nd Eng. in Med. and Biol. Conf., pp. 5250-5253, Sep. 2010. X. Huo, U.M. Jow, and M. Ghovanloo, “Radiation characterization of an intra-oral wireless device at multiple ISM bands: 433 MHz, 915 MHz, and 2.42 GHz,” Proc. IEEE 32nd Eng. in Med. and Biol. Conf., pp. 1425-1428, Sep. 2010. A.N. Johnson, X. Huo, C.W. Cheng, M. Ghovanloo, and M. Shinohara, “Effects of additional load on hand and tongue performance,” Proc. IEEE 32nd Eng. in Med. and Biol. Conf., pp. 6611-6614, Sep. 2010. X. Huo and M. Ghovanloo, “Evaluation of a wireless wearable tongue computer interface by individuals with high level spinal cord injuries,” Journal of Neural Engineering, vol. 7, #026008, Mar. 2010. C. Cheng, X. Huo, and M. Ghovanloo, “Towards a magnetic localization system for 3-D tracking of tongue movements in speech-language therapy,” Proc. IEEE 31st Eng. in Med. and Biol. Conf., pp. 563-566, Sep. 2009. X. Huo, C. Cheng, and M. Ghovanloo, “Evaluation of the tongue drive system by individuals with high-level spinal cord injury,” Proc. IEEE 31st Eng. in Med. and Biol. Conf., pp. 555-558, Sep. 2009. X. Huo and M. Ghovanloo, “Using unconstrained tongue motion as an alternative control surface for wheeled mobility,” IEEE Trans. on Biomed. Eng, vol. 56, no. 6, pp. 1719-1726, June 2009 X. Huo, J. Wang, and M. Ghovanloo, “Introduction and preliminary evaluation of tongue drive system: a wireless tongue-operated assistive technology for people with little or no upper extremity function,” Journal of Rehabilitation Research and Development, vol. 45, no. 6, pp. 921-938, Nov. 2008. X. Huo, J. Wang, and M. Ghovanloo, “A magneto-inductive sensor based wireless tongue-computer interface,” IEEE Trans. on Neural Sys. Rehab. Eng., vol. 16, no. 5, pp. 497-504, Oct. 2008. X. Huo, J. Wang, and M. Ghovanloo, “Wireless control of powered wheelchairs with tongue motion using tongue drive assistive technology,” Proc. IEEE 30th Eng. in Med. and Biol. Conf., pp. 4199-4202, Aug. 2008. J. Wang, X. Huo, and M. Ghovanloo, “A quadratic particle swarm optimization method for magnetic tracking of tongue motion in speech disorders,” Proc. IEEE 30th Eng. in Med. and Biol. Conf., pp. 4222-4225, Aug. 2008. X. Huo, J. Wang, and M. Ghovanloo, “Using Tongue Drive system as a new interface to control powered wheelchairs,” Proc. RESNA Conference, Washington, DC, June 2008. J. Wang, X. Huo, and M. Ghovanloo, “Tracking tongue movements for environment control using particle swarm optimization,” Proc. IEEE Intl. Symp. on Circuits and Systems, pp.1982-1985, May 2008. X. Huo, J. Wang, and M. Ghovanloo, “Using magneto-inductive sensors to detect tongue position in a wireless assistive technology for people with severe disabilities,” Proc. IEEE Sensors Conference, pp. 732-735, Oct. 2007 (First Place Poster Award). M. Ghovanloo, “Tongue operated assistive technologies,” Proc. IEEE 29th Eng. in Med. and Biol. Conf., pp. 4376-4379, Aug. 2007. X. Huo, J. Wang, and M. Ghovanloo, “A wireless tongue-computer interface using stereo differential magnetic field measurement,” Proc. IEEE 29th Eng. in Med. and Biol. Conf., pp. 5723-5726, Aug. 2007. X. Huo, J. Wang, and M. Ghovanloo, “Using magneto-inductive sensors to detect tongue position in a wireless assistive technology for people with severe disabilities,” To be presented at the IEEE Sensors Conference, Atlanta, GA, Oct. 2007. X. Huo, J. Wang, and M. Ghovanloo, “Use of tongue movements as a substitute for arm and hand functions in people with severe disabilities,” Proc. RESNA Conference, Phoenix, AZ, June 2007. X. Huo, J. Wang, and M. Ghovanloo, “A magnetic wireless tongue-computer interface,” Proc. 3rd Intl. IEEE/EMBS Conf. on Neural Engineering, pp. 322-326, May 2007. G. Krishnamurthy and M. Ghovanloo, “Tongue Drive: A tongue operated magnetic sensor based wireless assistive technology for people with severe disabilities,” IEEE Intl. Symp. on Circuits and Systems, pp. 5551-5554, May 2006. Text Box: I. Introduction Assistive technologies play a critical role in the lives of people with severe disabilities and help them to lead independent self-supportive lives. Persons severely disabled as a result of causes ranging from traumatic brain and spinal cord injuries to stroke and cerebral palsy generally find it extremely difficult to carry out everyday tasks without continuous help. Assistive technologies that help them communicate their intentions and effectively control their environment, especially to operate a computer, can greatly improve the quality of life for this group of people and may even help them to be employed.

© 2012 Maysam Ghovanloo
Text Box: II. Alternative Assistive Technologies A large group of assistive devices are available that are controlled by switches. The switch integrated hand splint, sip and puff device, chin control system, and electromyography (EMG) switch are all switch based systems and provide the user with limited degrees of freedom. A group of head-mounted assistive devices has been developed that emulate a computer mouse with head movements. Cursor movements in these devices are controlled by tracking an infrared beam emitted or reflected from a transmitter or reflector attached to the user’s glasses, cap, or headband. Tilt sensors and video-based computer interfaces that can track a facial feature have also been implemented. One limitation of these devices is that only those people whose head movement is not inhibited may avail of the technology. Another limitation is that the user’s head should always be in positions within the range of the device sensors. For example the controller may not be accessible when the user is lying in bed or not sitting in front of a computer. Another category of computer access systems operate by tracking eye movements from corneal reflections and pupil position. Electro-oculographic (EOG) potential measurements have also been used for detecting the eye movements. A major limitation of these devices is that they affect the users’ eyesight by requiring extra eye movements that can interfere with users’ normal visual activities such as reading, writing, and watching. The needs of persons with severe motor disabilities who cannot benefit from mechanical movements of any body organs are addressed by utilizing electric signals originated from brain waves or muscle twitches. Such brain computer interfaces (BCI), either invasive or noninvasive, have been the subject of major research activities over the last three decades. BCIs that operate based on electroencephalography (EEG) signals are very slow and limited in bandwidth. Implantable BCI technologies (single unit, LFP, or ECoG), on the other hand, are highly invasive (require a brain surgery) and heavily rely on signal processing and complex computational algorithms, which can results in delays and bulky systems that may also be very costly. Text Box: III. Why Tongue? Tongue and mouth occupy an amount of sensory and motor cortex that rivals that of the fingers and the hand. Hence they are inherently capable of sophisticated motor control and manipulation tasks. This is evident in vocalization and ingestion. The tongue is connected to the brain by the hypoglossal nerve, which generally escapes severe damage even in high level spinal cord injuries. Noninvasive access to the tongue is readily available. It is also the last to be affected in most neuromuscular degenerative disorders. The tongue has many degrees of freedom, and it can move very fast and accurately within the mouth cavity. One can touch every single tooth in his/her mouth with the tip of the tongue. Therefore it is a suitable organ for manipulating assistive devices. The tongue muscle has a very low rate of perceived exertion. Therefore, a tongue operated device can be used continuously over a long period. Also, a tongue-based device is hidden in the mouth and gives its user a certain degree of privacy. Text Box: V. Prototype Tongue Drive System We have developed several prototypes of the Tongue Drive System to evaluate the feasibility and performance of this approach in environmental control. The main purpose of this early prototype was to substitute mouse in computer access by moving the cursor on the computer screen based on the location of the magnetic tracer relative to four magnetic sensors. Four ratiometric linear sensors that were mounted in cavities created in a mouthguard. Text Box: VI. Substituting Mouse for Computer Access We have been able to successfully substitute mouse with the Tongue Drive System for computer access. Users can move the cursor on the screen using their tongue motions. They can also issue a single or double-click for selecting icons or opening folders. Several GUIs have been developed for the prototype Tongue Drive System. One of them is a simple computer game, called Fish Tales. Accessible entertainments are even more important in improving the quality of life for individuals with disabilities than their healthy counterparts. This experiment evaluates the usability of the Tongue Drive System in enabling users to play computer games, which are normally controlled by keyboard or mouse. In this GUI, payers use their tongues to navigate a red fish to catch the smaller fish, while avoiding being caught by the bigger fish. Moving the tongue in a certain direction, move the mouse cursor on the screen, resulting in the red fish swimming in that direction. The further the cursor is moved from the current location of the fish, the faster the fish swims. The goal is to catch as many smaller fish as possible. When the subject’s fish eats enough smaller fish, it grows and can eat larger fish.

Tongue Drive System Substituting  Mouse for Computer Access

Using  Tongue Drive System to Drive an Electric Wheelchair
Text Box: VII. Tongue Drive System for Wheelchair Control We have developed an interface that allows individuals to use the Tongue Drive System for controlling electric-powered wheelchairs by substituting the joystick. Similar to cursor movement on the computer screen, all that users need to do is to touch predefined positions in their mouth with the tip of their tongues to drive the wheelchair forward, backward, turn left, or turn right. They can accelerate the wheelchair by holding their tongues in the forward position, and decelerate by returning their tongues back to its resting position. They can even control their powered seating position for weight shifts by switching the wheelchair control mode from driving to seating control. Text Box: VIII. Clinical Testing and Evaluation After testing the Tongue Drive System by able-bodied subjects in a variety of experiments, we have now reached the stage that we can evaluate it in clinical settings by individuals with severe disabilities. We have successfully completed one round of clinical trials at the Shepherd Center in Atlanta, GA. In this trial, 13 individuals with spinal cord injury at C2-C5 level used the Tongue Drive System for computer access and wheelchair control, while the research team collected data on the usability and efficacy of the system hardware, signal processing algorithms, and GUI software. Participants also provided us with valuable feedback on how to improve the Tongue Drive System further and how they would like to use it. Second Clinical Trial (Ongoing): << Currently Recruiting! >> We are now recruiting for our second round of clinical trials which is ongoing at two major (Top10) rehabilitation centers across the United States: Shepherd Center in Atlanta, GA Rehabilitation Institute of Chicago, Chicago, IL Please see the details of this clinical trial and contact Dr. Ghovanloo if you are eligible and interested in participating in this trial, which involves receiving a cute little tongue piercing by our professional physicians! :-)

Team Tongue Drive with one of the participants in the 2nd TDS clinical trial 

First Clinical Trial at Shepherd Center
Text Box: Ongoing Clinical Trial Only for individuals with quadriplegia who live in Atlanta, GA or Chicago, IL, USA

Tongue Drive System V5

An early prototype of the internal Tongue Drive System (iTDS)

Magnetic Tongue Stud made of Titanium by Anatometal 

Text Box: Intraoral Tongue Drive System (iTDS) Technical info

Tongue Drive System in the NEWS:

 

Intra Tongue Drive System featured on cnet:

Now your tongue can secretly operate a computer, wheelchair

Intra Tongue Drive System featured on Georgia Tech research NEWS website:

Tongue Drive System Goes Inside the Mouth to Improve Performance and User Comfort

Tongue Drive featured in Spinal Column magazine:

Powered by the Tongue

Tongue Drive featured in the WGN-TV Medical Watch:

Tongue-Drive System

Tongue Drive featured in the WSB-TV:

New device may give new freedom to those in wheelchairs

Tongue Drive featured in the New York Times:

Piercing a Tongue, in the Name of Mobility

Tongue Drive System becomes a 2010 DaVinci Awards finalist and goes on to win the inaugural People’s Choice Award: “The LEO”

Tongue Drive featured in CNN Health Minute:

Wheelchair Mobility at the Tip of the Tongue 

Tongue Drive featured in e-Advances, the NIH-NIBIB online magazine:

Tongue-Operated Devices Help Paralyzed People

Tongue Drive featured on Science Nation (The NSF online Magazine):

Tongue Driver

Tongue Drive on Georgia Tech research NEWS website:

Clinical Trial Shows That Quadriplegics Can Use Tongue Drive System

Tongue Drive featured in New Scientist:

Magnet and glue turn tongue into joystick

Tongue Drive featured in USA TODAY:

'Tongue Drive' operates wheelchair

Tongue Drive featured in Reuters:

Device puts steering at the tip of the tongue