School of Electrical and Computer Engineering

Text Box: Tongue Drive System (TDS): 
A Brain-Tongue-Computer Interface
Text Box: IV. Tongue Drive System
Tongue Drive System (TDS) tracks the tongue motion by an array of magnetic sensors, which measure the magnetic field generated by a small permanent magnet, the size of a grain of rice, that is coated by a biocompatible material such as titanium and attached to the tongue via piercing, implantation, or adhesion. Magnetic sensors can be either mounted on a dental retainer and clipped on the teeth (internal Tongue Drive System or iTDS) or on a headset (external Tongue Drive System or eTDS) positioned near the cheeks. Sensor outputs are amplified, multiplexed, digitized, and transmitted wirelessly to an external controller unit. 
Signals received by the external controller, which can be a portable computer or a smartphone (e.g. iPhone) are processed to indicate the motion of the permanent magnet and consequently the tongue position within the oral cavity. We can assign a certain control function to each particular tongue movement in the software and customize the system for each individual user. These user-defined control functions may then be used to operate a variety of devices and equipments including computers, phones, and powered wheelchairs.
The signals from the magnetic sensors are linear functions of the magnetic field, which is a continuous position-dependent property. Thus a few sensors are able to capture a wide variety of tongue movements. This mechanism can provide a tremendous advantage over switch based devices in that the user will have the options of proportional, fuzzy, or adaptive control, which can offer smoother, faster, and more natural control over the environment.

© 2006 Maysam Ghovanloo

Internal Tongue Drive System (iTDS) consists of a permanent magnet attached to the tongue plus an array of magnetic sensors around the lower teeth on a dental retainer. 

Text Box: Related Publications:
H. Park and M. Ghovanloo, “Wireless communication with intraoral devices using off-the-shelf antennas,” Accepted for publication in IEEE Trans. on Microwave Theory and Techniques, Oct. 2014. 
H. Park and M. Ghovanloo, “An arch-shaped intraoral tongue drive system with built-in tongue-computer interfacing SoC,” Sensors, vol. 14, pp. 21565-21587, Nov. 2014, doi:10.3390/s141121565. 
A. Ayala-Acevedo and M. Ghovanloo, “Smartphone-compatible robust classification algorithm for the Tongue Drive System,” Proc. IEEE Biomedical Circuits and Systems Conf., pp. 161-164, Oct. 2014. 
J. Kim, H. Park, J. Bruce, D. Rowles, D. Pucci, J. Holbrook, B. Nardone, D.P. West, A.E. Laumann, E. Roth, E. Veledar, and M. Ghovanloo, “Qualitative assessment of tongue drive system by people with high-level spinal cord injury,” Journal of Rehabilitation Research and Development, vol. 51, no. 3, pp. 451–466, Aug. 2014. 
J. Kim, H. Park, J. Bruce, E. Sutton, D. Rowles, D. Pucci, J. Holbrook, J. Minocha, B. Nardone, D. West, A. Laumann, E. Roth, M. Jones, E. Veledar, and M. Ghovanloo, “Tongue enables computer and wheelchair access for the people with high-level disabilities,” Science Translational Medicine, vol. 5, no. 213, p. 213ra166, Nov. 2013. DOI: 10.1126/scitranslmed.3006296 Abstract, Full Text, Reprint (PDF)
J.S. Minocha, J.S. Holbrook, D.P. West, M. Ghovanloo, A. Laumann, “Development of a tongue-piercing method for use with assistive technology,” J. Am. Med. Association (JAMA) Dermatology, Published online jamadermatology.com, Nov. 27, 2013. 
X. Huo, A.N. Johnson-Long, M. Ghovanloo, M. Shinohara, “Motor performance of tongue with a computer integrated system under different levels of background physical exertion,” Ergonomics, vol. 56, no. 11, pp. 1733–1744, Sep. 2013.  
M. Kothari, P. Svensson, J. Jensen, T. Davidsen-Holm, M. Skorstengaard, Feldbek-Nielsen, M. Ghovanloo, L. Baad-Hansen, “Tongue controlled computer game: A new approach for rehabilitation of tongue motor function,” Archives of Physical Medicine and Rehabilitation, Oct. 2013, In press http://dx.doi.org/10.1016/j.apmr.2013.08.008 
X. Huo, H. Park, J. Kim, and M. Ghovanloo, “A dual-mode human computer interface combining speech and tongue motion for people with severe disabilities,” IEEE Trans. on Neural Sys. Rehab, vol. 21, no. 6, pp. 979-991, Nov. 2013.  
J. Kim, C. Bulach, K.M. Richards, D. Wu, A.J. Butler, and M. Ghovanloo, “An apparatus for improving upper limb function by engaging synchronous tongue motion,” IEEE Neural Engineering Conference, pp. 1574-1577, Nov. 2013. 
M. Kothari, P. Svensson, J. Jensen, A. Kjærsgaard, J. Kim, M. Ghovanloo, L. Baad-Hansen, “Training-induced cortical plasticity compared between three tongue-training paradigms,” Neuroscience, vol. 246, pp. 1–12, Aug. 2013, http://dx.doi.org/10.1016/j.neuroscience.2013.04.040
H. Park, M. Kiani, H.M. Lee, J. Kim, B. Gosselin, and M. Ghovanloo, “A wireless magnetoresistive sensing system for an intraoral tongue-computer interface,” IEEE Trans. on Biomed. Circuits and Systems, vol. 6, no. 6, pp. 571–585, Dec. 2012. 
X. Huo, H. Park, and M. Ghovanloo, “Dual-mode tongue drive system: using speech and tongue motion to improve computer access for people with disabilities,” Proc. Wireless Health Conf., pp. 47-54, Oct. 2012. 
X. Huo and M. Ghovanloo, “Tongue Drive: A wireless tongue-operated means for people with severe disabilities to communicate their intentions,” IEEE Communications Magazine, vol. 50, no. 10, pp. 128–135, Oct. 2012. 
H. Park, J. Kim, X. Huo, I. Hwang, and M. Ghovanloo, “Development and preliminary evaluation of an intraoral Tongue Drive System,” Proc. IEEE 34th Eng. in Med. and Biol. Conf., pp. 1157-1160, Sep. 2012.
A. Ayala-Acevedo and M. Ghovanloo, “Quantitative assessment of magnetic sensor signal processing algorithms in a wireless tongue-operated assistive technology,” Proc. IEEE 34th Eng. in Med. and Biol. Conf., pp. 3692-3695, Sep. 2012. 
J. Kim, H. Park, and M. Ghovanloo, “Tongue-operated assistive technology with access to common smartphone applications via Bluetooth link,” Proc. IEEE 34th Eng. in Med. and Biol. Conf., pp. 4054-4057, Sep. 2012. 
J. Kim, X. Huo, J. Minocha, J. Holbrook, A. Laumann, and M. Ghovanloo, “Evaluation of a smartphone platform as a wireless interface between Tongue Drive System and electric-powered wheelchairs,” IEEE Trans. Biomed. Eng., vol. 59, no. 6, pp. 1787–1796, June. 2012.
M. Kothari, P. Svensson, X. Huo, M. Ghovanloo, L. Baad-Hansen, “Force and complexity of tongue task training influences behavioral measures of motor learning,” European J. Oral Sciences, vol. 119, DOI: 10.1111/j.1600-0722.2011.00894.x, Dec. 2011. 
X. Huo, and M. Ghovanloo, “Using speech recognition to enhance the Tongue Drive System functionality in computer access,” Proc. IEEE 33rd Eng. in Med. and Biol. Conf., pp. 6393-6396, Sep. 2011.
B. Yousefi, X. Huo, and M. Ghovanloo, “Preliminary assessment of Tongue Drive System in medium term usage for computer access and wheelchair control,” Proc. IEEE 33rd Eng. in Med. and Biol. Conf., pp. 5766-5769, Sep. 2011.
E.B. Sadeghian, X. Huo, and M. Ghovanloo, “Command detection and classification in tongue drive assistive technology,” Proc. IEEE 33rd Eng. in Med. and Biol. Conf., pp. 5465-5468, Sep. 2011. 
H. Park, J. Kim, X. Huo, I. Hwang, and M. Ghovanloo, “New ergonomic headset for Tongue-Drive System with wireless smartphone interface,” Proc. IEEE 33rd Eng. in Med. and Biol. Conf., pp. 7344-7367, Sep. 2011. 
B. Yousefi, X. Huo, E. Veledar, and M. Ghovanloo, “Quantitative and comparative assessment of learning in a tongue-operated computer input device,” IEEE Trans. Info. Tech. in Biomedicine, vol. 15, no. 5, pp. 747-757, Sep. 2011.  
B. Yousefi, X. Huo, and M. Ghovanloo, “Using Fitts’s law for evaluating tongue drive system as a pointing device for computer access,” Proc. IEEE 32nd Eng. in Med. and Biol. Conf., pp. 4403-4406, Sep. 2010. 
J. Kim, X. Huo, and M. Ghovanloo, “Wireless control of smartphones with tongue motion using tongue drive assistive technology,” Proc. IEEE 32nd Eng. in Med. and Biol. Conf., pp. 5250-5253, Sep. 2010. 
X. Huo, U.M. Jow, and M. Ghovanloo, “Radiation characterization of an intra-oral wireless device at multiple ISM bands: 433 MHz, 915 MHz, and 2.42 GHz,” Proc. IEEE 32nd Eng. in Med. and Biol. Conf., pp. 1425-1428, Sep. 2010. 
A.N. Johnson, X. Huo, C.W. Cheng, M. Ghovanloo, and M. Shinohara, “Effects of additional load on hand and tongue performance,” Proc. IEEE 32nd Eng. in Med. and Biol. Conf., pp. 6611-6614, Sep. 2010.
X. Huo and M. Ghovanloo, “Evaluation of a wireless wearable tongue computer interface by individuals with high level spinal cord injuries,” Journal of Neural Engineering, vol. 7, #026008, Mar. 2010.
C. Cheng, X. Huo, and M. Ghovanloo, “Towards a magnetic localization system for 3-D tracking of tongue movements in speech-language therapy,” Proc. IEEE 31st Eng. in Med. and Biol. Conf., pp. 563-566, Sep. 2009.  
X. Huo, C. Cheng, and M. Ghovanloo, “Evaluation of the tongue drive system by individuals with high-level spinal cord injury,” Proc. IEEE 31st Eng. in Med. and Biol. Conf., pp. 555-558, Sep. 2009. 
X. Huo and M. Ghovanloo, “Using unconstrained tongue motion as an alternative control surface for wheeled mobility,” IEEE Trans. on Biomed. Eng, vol. 56, no. 6, pp. 1719-1726, June 2009 
X. Huo, J. Wang, and M. Ghovanloo, “Introduction and preliminary evaluation of tongue drive system: a wireless tongue-operated assistive technology for people with little or no upper extremity function,” Journal of Rehabilitation Research and Development, vol. 45, no. 6, pp. 921-938, Nov. 2008. 
X. Huo, J. Wang, and M. Ghovanloo, “A magneto-inductive sensor based wireless tongue-computer interface,” IEEE Trans. on Neural Sys. Rehab. Eng., vol. 16, no. 5, pp. 497-504, Oct. 2008. 
X. Huo, J. Wang, and M. Ghovanloo, “Wireless control of powered wheelchairs with tongue motion using tongue drive assistive technology,” Proc. IEEE 30th Eng. in Med. and Biol. Conf., pp. 4199-4202, Aug. 2008. 
J. Wang, X. Huo, and M. Ghovanloo, “A quadratic particle swarm optimization method for magnetic tracking of tongue motion in speech disorders,” Proc. IEEE 30th Eng. in Med. and Biol. Conf., pp. 4222-4225, Aug. 2008. 
X. Huo, J. Wang, and M. Ghovanloo, “Using Tongue Drive system as a new interface to control powered wheelchairs,” Proc. RESNA Conference, Washington, DC, June 2008.
J. Wang, X. Huo, and M. Ghovanloo, “Tracking tongue movements for environment control using particle swarm optimization,” Proc. IEEE Intl. Symp. on Circuits and Systems, pp.1982-1985, May 2008. 
X. Huo, J. Wang, and M. Ghovanloo, “Using magneto-inductive sensors to detect tongue position in a wireless assistive technology for people with severe disabilities,” Proc. IEEE Sensors Conference, pp. 732-735, Oct. 2007 (First Place Poster Award). 
M. Ghovanloo, “Tongue operated assistive technologies,” Proc. IEEE 29th Eng. in Med. and Biol. Conf., pp. 4376-4379, Aug. 2007. 
X. Huo, J. Wang, and M. Ghovanloo, “A wireless tongue-computer interface using stereo differential magnetic field measurement,” Proc. IEEE 29th Eng. in Med. and Biol. Conf., pp. 5723-5726, Aug. 2007. 
X. Huo, J. Wang, and M. Ghovanloo, “Using magneto-inductive sensors to detect tongue position in a wireless assistive technology for people with severe disabilities,” To be presented at the IEEE Sensors Conference, Atlanta, GA, Oct. 2007. 
X. Huo, J. Wang, and M. Ghovanloo, “Use of tongue movements as a substitute for arm and hand functions in people with severe disabilities,” Proc. RESNA Conference, Phoenix, AZ, June 2007. 
X. Huo, J. Wang, and M. Ghovanloo, “A magnetic wireless tongue-computer interface,” Proc. 3rd Intl. IEEE/EMBS Conf. on Neural Engineering, pp. 322-326, May 2007. 
G. Krishnamurthy and M. Ghovanloo, “Tongue Drive: A tongue operated magnetic sensor based wireless assistive technology for people with severe disabilities,” IEEE Intl. Symp. on Circuits and Systems, pp. 5551-5554, May 2006.
Text Box: I. Introduction
Assistive technologies play a critical role in the lives of people with severe disabilities and help them to lead independent self-supportive lives. Persons severely disabled as a result of causes ranging from traumatic brain and spinal cord injuries to stroke and cerebral palsy generally find it extremely difficult to carry out everyday tasks without continuous help. Assistive technologies that help them communicate their intentions and effectively control their environment, especially to operate a computer, can greatly improve the quality of life for this group of people and may even help them to be employed.

© 2014 Maysam Ghovanloo

Text Box: II. Alternative Assistive Technologies 
A large group of assistive devices are available that are controlled by switches. The switch integrated hand splint, sip and puff device, chin control system, and electromyography (EMG) switch are all switch based systems and provide the user with limited degrees of freedom. 
A group of head-mounted assistive devices has been developed that emulate a computer mouse with head movements. Cursor movements in these devices are controlled by tracking an infrared beam emitted or reflected from a transmitter or reflector attached to the user’s glasses, cap, or headband. Tilt sensors and video-based computer interfaces that can track a facial feature have also been implemented. One limitation of these devices is that only those people whose head movement is not inhibited may avail of the technology. Another limitation is that the user’s head should always be in positions within the range of the device sensors. For example the controller may not be accessible when the user is lying in bed or not sitting in front of a computer.  
Another category of computer access systems operate by tracking eye movements from corneal reflections and pupil position. Electro-oculographic (EOG) potential measurements have also been used for detecting the eye movements. A major limitation of these devices is that they affect the users’ eyesight by requiring extra eye movements that can interfere with users’ normal visual activities such as reading, writing, and watching. 
The needs of persons with severe motor disabilities who cannot benefit from mechanical movements of any body organs are addressed by utilizing electric signals originated from brain waves or muscle twitches. Such brain computer interfaces (BCI), either invasive or noninvasive, have been the subject of major research activities over the last three decades. BCIs that operate based on electroencephalography (EEG) signals are very slow and limited in bandwidth. Implantable BCI technologies (single unit, LFP, or ECoG), on the other hand, are highly invasive (require a brain surgery) and heavily rely on signal processing and complex computational algorithms, which can results in delays and bulky systems that may also be very costly. 
Text Box: III. Why Tongue?
Tongue and mouth occupy an amount of sensory and motor cortex that rivals that of the fingers and the hand. Hence they are inherently capable of sophisticated motor control and manipulation tasks. This is evident in vocalization and ingestion. The tongue is connected to the brain by the hypoglossal nerve, which generally escapes severe damage even in high level spinal cord injuries. Noninvasive access to the tongue is readily available. It is also the last to be affected in most neuromuscular degenerative disorders. The tongue has many degrees of freedom, and it can move very fast and accurately within the mouth cavity. One can touch every single tooth in his/her mouth with the tip of the tongue. Therefore it is a suitable organ for manipulating assistive devices. The tongue muscle has a very low rate of perceived exertion. Therefore, a tongue operated device can be used continuously over a long period. Also, a tongue-based device is hidden in the mouth and gives its user a certain degree of privacy.
Text Box: V. Prototype Tongue Drive System 
We have developed several prototypes of the Tongue Drive System to evaluate the feasibility and performance of this approach in environmental control. The main purpose of this early prototype was to substitute mouse in computer access by moving the cursor on the computer screen based on the location of the magnetic tracer relative to four magnetic sensors. Four ratiometric linear sensors that were mounted in cavities created in a mouthguard.  
Text Box: VI. Substituting Mouse for Computer Access
We have been able to successfully substitute mouse with the Tongue Drive System for computer access. Users can move the cursor on the screen using  their tongue motions. They can also issue a single or double-click for selecting icons or opening folders. Several GUIs have been developed for the prototype Tongue Drive System. One of them is a simple computer game, called Fish Tales. Accessible entertainments are even more important in improving the quality of life for individuals with disabilities than their healthy counterparts. This experiment evaluates the usability of the Tongue Drive System in enabling users to play computer games, which are normally controlled by keyboard or mouse. In this GUI, payers use their tongues to navigate a red fish to catch the smaller fish, while avoiding being caught by the bigger fish. Moving the tongue in a certain direction, move the mouse cursor on the screen, resulting in the red fish swimming in that direction. The further the cursor is moved from the current location of the fish, the faster the fish swims. The goal is to catch as many smaller fish as possible. When the subject’s fish eats enough smaller fish, it grows and can eat larger fish.

Tongue Drive System Substituting  Mouse for Computer Access

Using  Tongue Drive System to Drive an Electric Wheelchair

Text Box: VII. Tongue Drive System for Wheelchair Control
We have developed an interface that allows individuals to use the Tongue Drive System for controlling electric-powered wheelchairs by substituting the joystick. Similar to cursor movement on the computer screen, all that users need to do is to touch predefined positions in their mouth with the tip of their tongues to drive the wheelchair forward, backward, turn left, or turn right. They can accelerate the wheelchair by holding their tongues in the forward position, and decelerate by returning their tongues back to its resting position. They can even control their powered seating position for weight shifts by switching the wheelchair control mode from driving to seating control.
Text Box: VIII. Clinical Testing and Evaluation
After testing the Tongue Drive System by able-bodied subjects in a variety of experiments, we started evaluating it in a clinical settings by individuals with severe disabilities. We successfully completed the first round of clinical trial at the Shepherd Center in Atlanta, GA. In this trial, 13 individuals with spinal cord injury at C2-C5 level used the Tongue Drive System for computer access and wheelchair control, while the research team collected data on the usability and efficacy of the system hardware, signal processing algorithms, and GUI software. Participants also provided us with valuable feedback on how to improve the Tongue Drive System further and how they would like to use it.
In our Second Clinical Trial all participants, 23 able-bodied and 11 with tetraplegia in two cities, Atlanta and Chicago, received magnetic tongue piercings and used the system for 5 and 6 weeks, respectively. The results of this study has been published in the journal Science Translational Medicine.
Now we have focused on development of the intraoral Tongue Drive System (iTDS) to shrink the size of electronics and battery to the extent that it would comfortably fit inside the mouth without interfering with speech. The next step would be evaluating the iTDS in a clinical trial and comparing it with the eTDS in terms of usability, comfort, and acceptability among potential end users. We also plan to test both systems in the users’ home, office, and outdoor environments.

Team Tongue Drive with one of the participants in the 2nd TDS clinical trial 

First Clinical Trial at Shepherd Center

Text Box: Clinical Trial Completed 
If you have tetraplegia, live in Atlanta, GA or Chicago, IL, and interested in participating in the upcoming clinical trials, please do contact Dr. Ghovanloo.

Tongue Drive System V5

An early prototype of the internal Tongue Drive System (iTDS)

Magnetic Tongue Barbell made of Titanium by Anatometal 

Text Box: Intraoral Tongue Drive System (iTDS) Technical info

Tongue Drive System in the NEWS:

 

July 26, 2014:

Tongue Drive featured in New Scientist magazine: Hand to Mouth

July 24, 2014:

Tongue Drive featured in Narratively: The Tongue-Powered Test Drive

Georgia Tech coverage of the results of the TDS clinical study: Tongue Drive Wheelchair

Shepherd Center coverage of the results of the TDS clinical study: Tongue Piercing Put to Medical Use

Washington Post coverage of the results of the TDS clinical study: Tongue pierce lets the paralyzed drive wheelchairs

The wheelchair driven by a tongue piercing: Cool concept faces uphill battle

BBC coverage of the results of the TDS clinical study: Body piercing controls wheelchair

Radio Interview: BBC World Business Report

Fox NEWS coverage of the results of the TDS clinical study: Tongue-controlled wheelchairs prove effective for quadriplegic patients

Live Science coverage of the results of the TDS clinical study: Tongue-Controlled Wheelchair Helps Paralyzed People Move

Results of the TDS clinical study is published in Science Translational Medicine (Impact factor > 10).

Intra Tongue Drive System featured on cnet:

Now your tongue can secretly operate a computer, wheelchair

Intra Tongue Drive System featured on Georgia Tech research NEWS website:

Tongue Drive System Goes Inside the Mouth to Improve Performance and User Comfort

Tongue Drive featured in Spinal Column magazine:

Powered by the Tongue

Tongue Drive featured in the WGN-TV Medical Watch: Tongue-Drive System

Tongue Drive featured in the New York Times:

Piercing a Tongue, in the Name of Mobility

Tongue Drive System becomes a 2010 DaVinci Awards finalist and goes on to win the inaugural People’s Choice Award: “The LEO”

Tongue Drive featured in CNN Health Minute:

Wheelchair Mobility at the Tip of the Tongue 

Tongue Drive featured in e-Advances, the NIH-NIBIB online magazine:

Tongue-Operated Devices Help Paralyzed People

Tongue Drive featured on Science Nation (The NSF online Magazine):

Tongue Driver

Tongue Drive on Georgia Tech research NEWS website:

Clinical Trial Shows That Quadriplegics Can Use Tongue Drive System

Tongue Drive featured in New Scientist:

Magnet and glue turn tongue into joystick

Tongue Drive featured in USA TODAY:

'Tongue Drive' operates wheelchair

Tongue Drive featured in Reuters:

Device puts steering at the tip of the tongue