Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Imprecise but Fun: Playful Interaction Using Electromyography

Published: 20 September 2022 Publication History

Abstract

Novel input methods for game design often excite users, especially if they extend the way one interacts with the system. Electromyography (EMG) has the inherent potential to provide an intuitive - yet challenging - input channel for interactive systems. While this difficulty in control often limits the scope of applications for EMG in most systems, we argue that these qualities are especially relevant for games and playful interaction. The inherently challenging qualities of EMG input make the modality a prime candidate for designing body-centric playful experiences. Yet, we still need to understand its limitations to create engaging rather than frustrating experiences for users. In this work, we investigate EMG's potential to support playful interaction through exploratory studies, deriving feasible game interactions based on EMG's technical constraints, and study their application in game design. Based on our findings, we highlight design implications and pitfalls to avoid when creating EMG-based entertainment systems.

Supplementary Material

ZIP File (v6mhci190aux.zip)
- emg_mounting_manual: guide explaining how to mount the electrodes and connect the EMG device - pilot_questionnaire_results: detailed results for the questionnaire in the pilot study (same questions as used in Study I, see paper) - playful_emg: video and corresponding subtitles of the submission
MP4 File (v6mhci190.mp4)
Supplemental video

References

[1]
Ashu Adhikari, Abraham M. Hashemian, Thinh Nguyen-Vo, Ernst Kruijff, Markus von der Heyde, and Bernhard E. Riecke. 2021. Lean to Fly : Leaning-Based Embodied Flying Can Improve Performance and User Experience in 3D Navigation. Frontiers in Virtual Reality, Vol. 2 (2021). https://www.frontiersin.org/article/10.3389/frvir.2021.730334
[2]
Christoph Amma, Thomas Krings, Jonas Böer, and Tanja Schultz. 2015. Advancing Muscle-Computer Interfaces with High-Density Electromyography. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 929--938. https://doi.org/10.1145/2702123.2702501
[3]
D. T. Barry, K. E. Gordon, and G. G. Hinton. 1990. Acoustic and Surface EMG Diagnosis of Pediatric Muscle Disease. Muscle Nerve, Vol. 13, 4 (April 1990), 286--290. https://doi.org/10.1002/mus.880130403
[4]
Enrico Costanza, Samuel A. Inverso, and Rebecca Allen. 2005. Toward Subtle Intimate Interfaces for Mobile Devices Using an EMG Controller. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '05). ACM, New York, NY, USA, 481--489. https://doi.org/10.1145/1054972.1055039
[5]
Enrico Costanza, Samuel A. Inverso, Rebecca Allen, and Pattie Maes. 2007. Intimate Interfaces in Action : Assessing the Usability and Subtlety of Emg-based Motionless Gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, New York, NY, USA, 819--828. https://doi.org/10.1145/1240624.1240747
[6]
Kate C. Ewing, Stephen H. Fairclough, and Kiel Gilleade. 2016. Evaluation of an Adaptive Game That Uses EEG Measures Validated during the Design Process as Inputs to a Biocybernetic Loop. Front Hum Neurosci, Vol. 10 (May 2016), 223. https://doi.org/10.3389/fnhum.2016.00223
[7]
Florian 'Floyd' Mueller, Rakesh Patibanda, Richard Byrne, Zhuying Li, Yan Wang, Josh Andres, Xiang Li, Jonathan Marquez, Stefan Greuter, Jonathan Duckworth, and Joe Marshall. 2021. Limited Control Over the Body as Intriguing Play Design Resource. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). Association for Computing Machinery, New York, NY, USA, 1--16. https://doi.org/10.1145/3411764.3445744
[8]
Yue Gao and Regan L Mandryk. 2011. GrabApple: The design of a casual exergame. In International Conference on Entertainment Computing. Springer, 35--46.
[9]
Nadia Garcia-Hernandez, Karen Garza-Martinez, Vicente Parra-Vega, Armando Alvarez-Sanchez, and Laura Conchas-Arteaga. 2019. Development of an EMG-based exergaming system for isometric muscle training and its effectiveness to enhance motivation, performance and muscle strength. International Journal of Human-Computer Studies, Vol. 124 (2019), 44--55.
[10]
Mohammad Ghassemi, Kristen Triandafilou, Alex Barry, Mary Ellen Stoykov, Elliot Roth, Ferdinando A. Mussa-Ivaldi, Derek G. Kamper, and Rajiv Ranganathan. 2019. Development of an EMG-Controlled Serious Game for Rehabilitation. IEEE Trans Neural Syst Rehabil Eng, Vol. 27, 2 (Feb. 2019), 283--292. https://doi.org/10.1109/TNSRE.2019.2894102
[11]
R. a. R. C. Gopura and Kazuo Kiguchi. 2012. Application of Surface Electromyographic Signals to Control Exoskeleton Robots. IntechOpen. https://doi.org/10.5772/26389
[12]
Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index ): Results of Empirical and Theoretical Research. In Human Mental Workload. North-Holland, Oxford, England, 139--183. https://doi.org/10.1016/S0166--4115(08)62386--9
[13]
A. Hernandez Arieta, R. Katoh, H. Yokoi, and Y. Wenwei. 2006. Development of a Multi-DOF Electromyography Prosthetic System Using the Adaptive Joint Mechanism. https://doi.org/10.1533/abbi.2005.0060
[14]
Cedric Honnet, Hannah Perner-Wilson, Marc Teyssier, Bruno Fruchard, Jürgen Steimle, Ana C. Baptista, and Paul Strohmeier. 2020. PolySense : Augmenting Textiles with Electrical Functionality Using In-Situ Polymerization. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, Honolulu, HI, USA, 1--13. https://doi.org/10.1145/3313831.3376841
[15]
Donny Huang, Xiaoyi Zhang, T. Scott Saponas, James Fogarty, and Shyamnath Gollakota. 2015. Leveraging Dual-Observable Input for Fine-Grained Thumb Interaction Using Forearm EMG. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15). Association for Computing Machinery, New York, NY, USA, 523--528. https://doi.org/10.1145/2807442.2807506
[16]
Alayna Hughes and Sergi Jorda. 2021. Applications of Biological and Physiological Signals in Commercial Video Gaming and Game Research : A Review. Frontiers in Computer Science, Vol. 3 (2021). https://www.frontiersin.org/article/10.3389/fcomp.2021.557608
[17]
Richard J. Jagacinski, Daniel W. Repperger, Martin S. Moran, Sharon L. Ward, and Betty Glass. 1980. Fitts' Law and the Microstructure of Rapid Discrete Movements. Journal of Experimental Psychology: Human Perception and Performance, Vol. 6, 2 (1980), 309--320. https://doi.org/10.1037/0096--1523.6.2.309
[18]
Jakob Karolus, Francisco Kiss, Caroline Eckerth, Nicolas Viot, Felix Bachmann, Albrecht Schmidt, and Paweł W. Wo'zniak. 2021. EMBody : A Data-Centric Toolkit for EMG-Based Interface Prototyping and Experimentation. Proc. ACM Hum.-Comput. Interac., Vol. 5, EICS (2021), 195:1--195:29. https://doi.org/10.1145/3457142
[19]
Jakob Karolus, Hendrik Schuff, Thomas Kosch, Paweł W. Wozniak, and Albrecht Schmidt. 2018. EMGuitar : Assisting Guitar Playing with Electromyography. In Proceedings of the 2018 Designing Interactive Systems Conference (DIS '18). Association for Computing Machinery, Hong Kong, China, 651--655. https://doi.org/10.1145/3196709.3196803
[20]
Frederic Kerber, Michael Puhl, and Antonio Krüger. 2017. User-Independent Real-Time Hand Gesture Recognition Based on Surface Electromyography. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17). Association for Computing Machinery, New York, NY, USA, 1--7. https://doi.org/10.1145/3098279.3098553
[21]
J.F. Knight, C. Baber, A. Schwirtz, and H.W. Bristow. 2002. The Comfort Assessment of Wearable Computers. In Proceedings. Sixth International Symposium on Wearable Computers,. 65--72. https://doi.org/10.1109/ISWC.2002.1167220
[22]
John W. Krakauer and Reza Shadmehr. 2006. Consolidation of Motor Memory. Trends Neurosci., Vol. 29, 1 (Jan. 2006), 58--64. https://doi.org/10.1016/j.tins.2005.10.003
[23]
Dennis Krupke, Paul Lubos, Lena Demski, Jonas Brinkhoff, Gregor Weber, Fabian Willke, and Frank Steinicke. 2016. Control Methods in a Supernatural Flight Simulator. In 2016 IEEE Virtual Reality (VR ). 329--329. https://doi.org/10.1109/VR.2016.7504788
[24]
G.M. Lyons, P. Sharma, M. Baker, S. O'Malley, and A. Shanahan. 2003. A Computer Game-Based EMG Biofeedback System for Muscle Rehabilitation. In Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No. 03CH37439 ), Vol. 2. 1625--1628 Vol.2. https://doi.org/10.1109/IEMBS.2003.1279682
[25]
Regan L. Mandryk, M. Stella Atkins, and Kori M. Inkpen. 2006. A Continuous and Objective Evaluation of Emotional Experience with Interactive Play Environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06). ACM, New York, NY, USA, 1027--1036. https://doi.org/10.1145/1124772.1124926
[26]
Joe Marshall, Alexandru Dancu, and Florian "Floyd" Mueller. 2016. Interaction in Motion: Designing Truly Mobile Interaction. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (Brisbane, QLD, Australia) (DIS '16). Association for Computing Machinery, New York, NY, USA, 215--228. https://doi.org/10.1145/2901790.2901844
[27]
Soroosh Mashal, Gerold Hoelzl, and Matthias Kranz. 2019. Valkyrie Project : Flying Immersion in Virtual Reality. In 2019 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games ). 1--4. https://doi.org/10.1109/VS-Games.2019.8864545
[28]
Daniel Medeiros, Mauricio Sousa, Alberto Raposo, and Joaquim Jorge. 2020. Magic Carpet : Interaction Fidelity for Flying in VR. IEEE Transactions on Visualization and Computer Graphics, Vol. 26, 9 (Sept. 2020), 2793--2804. https://doi.org/10.1109/TVCG.2019.2905200
[29]
Roberto Merletti and Dario Farina. 2016. Surface Electromyography : Physiology, Engineering and Applications. John Wiley & Sons.
[30]
Florian Mueller, Stefan Agamanolis, and Rosalind Picard. 2003. Exertion Interfaces: Sports over a Distance for Social Bonding and Fun. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI '03). Association for Computing Machinery, New York, NY, USA, 561--568. https://doi.org/10.1145/642611.642709
[31]
Lennart Erik Nacke, Michael Kalyn, Calvin Lough, and Regan Lee Mandryk. 2011. Biofeedback Game Design: Using Direct and Indirect Physiological Control to Enhance Game Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). Association for Computing Machinery, New York, NY, USA, 103--112. https://doi.org/10.1145/1978942.1978958
[32]
Jakob Nielsen. 1993. Usability Engineering. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
[33]
Anton Nijholt, Danny Plass-Oude Bos, and Boris Reuderink. 2009. Turning Shortcomings into Challenges: Brain textendash Computer Interfaces for Games. Entertainment Computing, Vol. 1, 2 (April 2009), 85--94. https://doi.org/10.1016/j.entcom.2009.09.007
[34]
Aditya Shekhar Nittala, Arshad Khan, Klaus Kruttwig, Tobias Kraus, and Jürgen Steimle. 2020a. PhysioSkin : Rapid Fabrication of Skin-Conformal Physiological Interfaces. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--10. https://doi.org/10.1145/3313831.3376366
[35]
Aditya Shekhar Nittala, Arshad Khan, Klaus Kruttwig, Tobias Kraus, and Jürgen Steimle. 2020b. PhysioSkin : Rapid Fabrication of Skin-Conformal Physiological Interfaces. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, Honolulu, HI, USA, 1--10. https://doi.org/10.1145/3313831.3376366
[36]
D. Peleg, E. Braiman, E. Yom-Tov, and G. F. Inbar. 2002. Classification of Finger Activation for Use in a Robotic Prosthesis Arm. IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 10, 4 (Dec. 2002), 290--293. https://doi.org/10.1109/TNSRE.2002.806831
[37]
Monica Perusquía-Hernández, Saho Ayabe-Kanamura, Kenji Suzuki, and Shiro Kumano. 2019. The Invisible Potential of Facial Electromyography : A Comparison of EMG and Computer Vision When Distinguishing Posed from Spontaneous Smiles. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--9. https://doi.org/10.1145/3290605.3300379
[38]
M. Perusquía-Hernández, T. Enomoto, T. Martins, M. Otsuki, H. Iwata, and K. Suzuki. 2017. Embodied Interface for Levitation and Navigation in a 3D Large Space. In Proceedings of the 8th Augmented Human International Conference (AH '17). Association for Computing Machinery, New York, NY, USA, 1--9. https://doi.org/10.1145/3041164.3041173
[39]
Angkoon Phinyomark, Pornchai Phukpattaranont, and Chusak Limsakul. 2012. Feature Reduction and Selection for EMG Signal Classification. Expert Systems with Applications, Vol. 39, 8 (June 2012), 7420--7431. https://doi.org/10.1016/j.eswa.2012.01.102
[40]
Angkoon Phinyomark and Erik Scheme. 2018. EMG Pattern Recognition in the Era of Big Data and Deep Learning. BDCC, Vol. 2, 3 (Aug. 2018), 21. https://doi.org/10.3390/bdcc2030021
[41]
Alejandro Lopez Rincon, Hiroshi Yamasaki, and Shingo Shimoda. 2016. Design of a Video Game for Rehabilitation Using Motion Capture, EMG Analysis and Virtual Reality. In 2016 International Conference on Electronics, Communications and Computers (CONIELECOMP ). 198--204. https://doi.org/10.1109/CONIELECOMP.2016.7438575
[42]
Yvonne Rogers. 2006. Moving on from Weiser 's Vision of Calm Computing : Engaging UbiComp Experiences. In UbiComp 2006: Ubiquitous Computing (Lecture Notes in Computer Science ), Paul Dourish and Adrian Friday (Eds.). Springer, Berlin, Heidelberg, 404--421. https://doi.org/10.1007/11853565_24
[43]
Wataru Sakoda, Ramin Tadayon, Yusuke Kishishita, Masataka Yamamoto, and Yuichi Kurita. 2020. Ski exergame for squat training to change load based on predicted Locomotive risk level. In 2020 IEEE/SICE International Symposium on System Integration (SII). IEEE, 289--294.
[44]
T Scott Saponas, Desney S. Tan, Dan Morris, and Ravin Balakrishnan. 2008. Demonstrating the Feasibility of Using Forearm Electromyography for Muscle-computer Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 515--524. https://doi.org/10.1145/1357054.1357138
[45]
T. Scott Saponas, Desney S. Tan, Dan Morris, Jim Turner, and James A. Landay. 2010. Making Muscle-computer Interfaces More Practical. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 851--854. https://doi.org/10.1145/1753326.1753451
[46]
E. Scheme and K. Englehart. 2014. On the Robustness of EMG Features for Pattern Recognition Based Myoelectric Control; A Multi-Dataset Comparison. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, Chicago, IL, 650--653. https://doi.org/10.1109/EMBC.2014.6943675
[47]
Mel Slater, Martin Usoh, and Anthony Steed. 1995. Taking Steps: The Influence of a Walking Technique on Presence in Virtual Reality. ACM Trans. Comput.-Hum. Interact., Vol. 2, 3 (Sept. 1995), 201--219. https://doi.org/10.1145/210079.210084
[48]
Ludger van Dijk, Corry K. van der Sluis, Hylke W. van Dijk, and Raoul M. Bongers. 2016. Learning an EMG Controlled Game : Task-Specific Adaptations and Transfer. PLoS One, Vol. 11, 8 (Aug. 2016), e0160817. https://doi.org/10.1371/journal.pone.0160817
[49]
Scott R. Vrana. 1993. The Psychophysiology of Disgust: Differentiating Negative Emotional Contexts with Facial EMG. Psychophysiology, Vol. 30, 3 (May 1993), 279--286. https://doi.org/10.1111/j.1469--8986.1993.tb03354.x
[50]
Pawel W. Wozniak, Jakob Karolus, Florian Lang, Caroline Eckerth, Johannes Schöning, Yvonne Rogers, and Jasmin Niess. 2021. Creepy Technology: What Is It and How Do You Measure It?. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan, May 08--13, 2021. ACM, 13. https://doi.org/10.1145/3411764.3445299
[51]
Yaying Zhang, Bernhard E. Riecke, Thecla Schiphorst, and Carman Neustaedter. 2019. Perch to Fly : Embodied Virtual Reality Flying Locomotion with a Flexible Perching Stance. In Proceedings of the 2019 on Designing Interactive Systems Conference (DIS '19). Association for Computing Machinery, New York, NY, USA, 253--264. https://doi.org/10.1145/3322276.3322357

Cited By

View all
  • (2024)Sonic Entanglements with Electromyography: Between Bodies, Signals, and RepresentationsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661572(2691-2707)Online publication date: 1-Jul-2024
  • (2024)Improving Electromyographic Muscle Response Times through Visual and Tactile Prior Stimulation in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642091(1-17)Online publication date: 11-May-2024
  • (2024)Understanding the influence of confounding factors in myoelectric control for discrete gesture recognitionJournal of Neural Engineering10.1088/1741-2552/ad491521:3(036015)Online publication date: 17-May-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction
Proceedings of the ACM on Human-Computer Interaction  Volume 6, Issue MHCI
MHCI
September 2022
852 pages
EISSN:2573-0142
DOI:10.1145/3564624
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 September 2022
Published in PACMHCI Volume 6, Issue MHCI

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. electromyography
  2. physiological interaction
  3. playful interaction

Qualifiers

  • Research-article

Funding Sources

  • H2020 European Research Council

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)63
  • Downloads (Last 6 weeks)10
Reflects downloads up to 20 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Sonic Entanglements with Electromyography: Between Bodies, Signals, and RepresentationsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661572(2691-2707)Online publication date: 1-Jul-2024
  • (2024)Improving Electromyographic Muscle Response Times through Visual and Tactile Prior Stimulation in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642091(1-17)Online publication date: 11-May-2024
  • (2024)Understanding the influence of confounding factors in myoelectric control for discrete gesture recognitionJournal of Neural Engineering10.1088/1741-2552/ad491521:3(036015)Online publication date: 17-May-2024
  • (2023)Toward Optimized VR/AR Ergonomics: Modeling and Predicting User Neck Muscle ContractionACM SIGGRAPH 2023 Conference Proceedings10.1145/3588432.3591495(1-12)Online publication date: 23-Jul-2023
  • (2023)A Survey on Measuring Cognitive Workload in Human-Computer InteractionACM Computing Surveys10.1145/358227255:13s(1-39)Online publication date: 13-Jul-2023
  • (2023)Tailor Twist: Assessing Rotational Mid-Air Interactions for Augmented RealityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581461(1-14)Online publication date: 19-Apr-2023
  • (2023)LibEMG: An Open Source Library to Facilitate the Exploration of Myoelectric ControlIEEE Access10.1109/ACCESS.2023.330454411(87380-87397)Online publication date: 2023

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media