research-article

Multi modal gesture identification for HCI using surface EMG

Authors:

Ganesh R. Naik,

Dinesh K. Kumar,

Sridhar P. ArjunanAuthors Info & Claims

MindTrek '08: Proceedings of the 12th international conference on Entertainment and media in the ubiquitous era

Pages 90 - 94

https://doi.org/10.1145/1457199.1457219

Published: 07 October 2008 Publication History

Abstract

Gesture and Speech comprise the most important modalities of human interaction. There has been a considerable amount of research attempts at incorporating these modalities for natural HCI. This involves challenge ranging from the low level signal processing of multi-modal input to the high level interpretation of natural speech and gesture in HCI. This paper proposes novel methods to recognize the hand gestures and unvoiced utterances using surface Electromyogram (sEMG) signals originating from different muscles. The focus of this work is to establish a simple, yet robust system that can be integrated to identify subtle complex hand gestures and unvoiced speech commands for control of prosthesis and other computer assisted devices. The proposed multi-modal system is able to identify the hand gestures and silent utterances using Independent Component Analysis (ICA) and Integral RMS (IRMS) of sEMG respectively. Training of the sEMG features was done using a designed ANN architecture and the results reported with overall recognition accuracy of 90.33%.

References

[1]

Chan, D. C., Englehart, K., Hudgins, B., and Lovely, D. F. 2002. A multi-expert speech recognition system using acoustic and myoelectric signals. In Proceedings of the 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society EMBS/BMES, Canada, 72--73.

[2]

Chan, A. D. C., and Englehart, K. B. 2005. Continuous myoelectric control for powered prostheses using hidden markov models. IEEE Trans. Biomed. Engg. 52, 1 (2005), 121--124.

[3]

Fistre, J., and Tanaka, A. 2002. Real time EMG gesture recognition for consumer electronics device control. Presented at Sony CSL Paris Open House, Paris, France. http://www.csl.sony.fr/atau/gesture/.

[4]

Fridlund, A. J., and Cacioppo, J. T. 1986. Guidelines for Human Electromyographic research. J. Psych. 23, 4 (1986), 567--589.

[5]

Hyvarinen, A., Karhunen, J., and Oja, E. 2001. Independent Component Analysis. John Wiley, New York.

[6]

Kermani, M. Z., Wheeler, B. C., Badie, K., and Hashemi, R. M. 1995. EMG Feature Evaluation for Movement Control of Upper Extremity Prostheses, IEEE Trans. Rehab. Engg. 2 (1995), 1267--1271.

[7]

Kumar, S., Kumar, D. K., Alemu, M., and Burry, M. 2004. EMG based voice recognition. In Proceedings of Sensor Networks and Information Processing. Melbourne, Australia.

[8]

Lapatki, G., Stegeman, D. F. and Jonas, I. E. 2003. A surface EMG electrode for the simultaneous observation of multiple facial muscles. J. Neuro. 123, 2 (2003), 117--128.

[9]

Manabe, H., Hiraiwa, A., and Sugimura, T. 2003. Unvoiced speech recognition using SEMG Mime Speech Recognition. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM Press, Florida, USA, 794--795.

Digital Library

[10]

Parsons, T. W. 1986. Voice and speech processing.

[11]

Pavlovic, V. I., Sharma, R., and Huang, T. S. 1997. Visual interpretation of hand gestures for human computer in--teraction. IEEE Trans. Pattern Analys. Machine Intell. 19 (2005), 677--695.

Digital Library

[12]

Rehg, J. M. and Kanade, T. 1996. Vision-based hand tracking for human-computer interaction. In Proceedings of the IEEE Workshop on Motion of Non-Rigid and Articulated Objects, 16--22.

[13]

Schlenzig, J., Hunter, E., and Jain, R. 1994. Vision based hand gesture interpretation using recursive estimation. In Proceedings of Twenty-eighth Asilomar Conference on Signals, Systems and Computers, 1267--1271.

[14]

Trejo, L. J., Wheeler, K. R., Jorgensen, C. C., Rosipal, R., Clanton, T. S., Matthews, B., Hibbs, A. D., Matthews, R., and Krupka, M. 2003. Multimodal neuroelectric interface development. IEEE Trans. Neural Sys. Rehab. Engg. 11, 2 (2003), 199--204.

[15]

Wheeler, K. R., and Jorgensen, C. C. 2003. Gestures as input: Neuroelectric joysticks and keyboards. IEEE Per. Comput. 2, 2 (2003), 56--61.

Digital Library

Cited By

Pohl HMuresan AHornbæk KBrewster SFitzpatrick GCox AKostakos V(2019)Charting Subtle Interaction in the HCI LiteratureProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300648(1-15)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300648
Fang JXu BZhou XQi H(2018)Research on gesture recognition based on sEMG and inertial sensor fusion2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC)10.1109/IAEAC.2018.8577542(1562-1567)Online publication date: Oct-2018
https://doi.org/10.1109/IAEAC.2018.8577542
Wang ZShi Y(2017)Daily behavior identification based on sEMG10.1063/1.4992824(020007)Online publication date: 2017
https://doi.org/10.1063/1.4992824
Show More Cited By

Index Terms

Multi modal gesture identification for HCI using surface EMG
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Assessing the Applicability of Surface EMG to Tongue Gesture Detection
IberSPEECH 2014: Proceedings of the Second International Conference on Advances in Speech and Language Technologies for Iberian Languages - Volume 8854

The most promising approaches for surface Electromyography EMG based speech interfaces commonly focus on the tongue muscles. Despite the interesting results in small vocabularies tasks, it is yet unclear which articulation gestures these sensors are ...
Surface Electromyographic Activity of Extrinsic Laryngeal Muscles in Cantonese Tone Production

Patients after total laryngectomy lose their ability to speak. Electrolarynx is a commonly used electronic device that helps these patients to verbally communicate. However, existing electrolarynx systems do not provide pitch control function, which is ...
Subtle Hand Gesture Identification for HCI Using Temporal Decorrelation Source Separation BSS of Surface EMG
DICTA '07: Proceedings of the 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications

Hand gesture identification has various human computer interaction (HCI) applications. This paper presents a method for subtle hand gesture identification from sEMG of the forearm by decomposing the signal into components originating from different ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MindTrek '08: Proceedings of the 12th international conference on Entertainment and media in the ubiquitous era

October 2008

221 pages

ISBN:9781605581972

DOI:10.1145/1457199

Conference Chairs:
Frans Mäyrä
University of Tampere, Finland
,
Katri Lietsala
University of Tampere, Finland
,
Heljä Franssila
University of Tampere, Finland
,
Editors:
Artur Lugmayr,
Frans Mäyrä,
Heljä Franssila,
Katri Lietsala,
General Chair:
Artur Lugmayr
Tampere University of Technology, Finland

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 October 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MindTrek08

Sponsor:

MindTrek08: MindTrek08

October 7 - 9, 2008

Tampere, Finland

Acceptance Rates

Overall Acceptance Rate 110 of 207 submissions, 53%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
537
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)1

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pohl HMuresan AHornbæk KBrewster SFitzpatrick GCox AKostakos V(2019)Charting Subtle Interaction in the HCI LiteratureProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300648(1-15)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300648
Fang JXu BZhou XQi H(2018)Research on gesture recognition based on sEMG and inertial sensor fusion2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC)10.1109/IAEAC.2018.8577542(1562-1567)Online publication date: Oct-2018
https://doi.org/10.1109/IAEAC.2018.8577542
Wang ZShi Y(2017)Daily behavior identification based on sEMG10.1063/1.4992824(020007)Online publication date: 2017
https://doi.org/10.1063/1.4992824
Duarte LCarriço LIsbister KMueller FBernhaupt R(2012)Power me Up!Proceedings of the 4th International Conference on Fun and Games10.1145/2367616.2367623(55-63)Online publication date: 4-Sep-2012
https://dl.acm.org/doi/10.1145/2367616.2367623
Al-Mulla MSepulveda FColley M(2011)An Autonomous Wearable System for Predicting and Detecting Localised Muscle FatigueSensors10.3390/s11020154211:2(1542-1557)Online publication date: 27-Jan-2011
https://doi.org/10.3390/s110201542
Duarte LCarriço LRomão TCorreia NInami MDias EChambel TKato HPrada RTerada T(2011)The influence of performance-oriented widgets on interactive behavior while playing videogamesProceedings of the 8th International Conference on Advances in Computer Entertainment Technology10.1145/2071423.2071491(1-8)Online publication date: 8-Nov-2011
https://dl.acm.org/doi/10.1145/2071423.2071491
Fairclough S(2011)Physiological Computing: Interfacing with the Human Nervous SystemSensing Emotions10.1007/978-90-481-3258-4_1(1-20)Online publication date: 20-Jul-2011
https://doi.org/10.1007/978-90-481-3258-4_1
Cheng JChen XLu ZWang KShen MGao WLee CYang JChen XEskenazi MZhang Z(2010)Key-press gestures recognition and interaction based on SEMG signalsInternational Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction10.1145/1891903.1891950(1-4)Online publication date: 8-Nov-2010
https://dl.acm.org/doi/10.1145/1891903.1891950

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents