research-article

Predicting Grasps with a Wearable Inertial and EMG Sensing Unit for Low-Power Detection of In-Hand Objects

Authors:

Philipp M. Scholl,

Kristof Van LaerhovenAuthors Info & Claims

AH '16: Proceedings of the 7th Augmented Human International Conference 2016

Article No.: 4, Pages 1 - 8

https://doi.org/10.1145/2875194.2875207

Published: 25 February 2016 Publication History

Abstract

Detecting the task at hand can often be improved when it is also known what object the user is holding. Several sensing modalities have been suggested to identify handheld objects, from wrist-worn RFID readers to cameras. A critical obstacle to using such sensors, however, is that they tend to be too power hungry for continuous usage. This paper proposes a system that detects grasping using first inertial sensors and then Electromyography (EMG) on the forearm, to then selectively activate the object identification sensors. This three-tiered approach would therefore only attempt to identify in-hand objects once it is known a grasping has occurred. Our experiments show that high recall can be obtained for grasp detection, 95% on average across participants, with the grasping of lighter and smaller objects clearly being more difficult.

References

[1]

F. Ahmad and P. Musilek. A keystroke and pointer control input interface for wearable computers. In Fourth Annual IEEE Intl. Conf. on Pervasive Computing and Communications, pages 2--11, 2006.

Digital Library

[2]

O. Amft, H. Junker, P. Lukowicz, G. Troester, and C. Schuster. Sensing muscle activities with body-worn sensors. In BSN 2006: Proceedings of the International Workshop on Wearable and Implantable Body Sensor Networks. IEEE press, 2006.

Digital Library

[3]

E. Berlin, J. Liu, K. Van Laerhoven, and B. Schiele. Coming to grips with the objects we grasp: Detecting interactions with efficient wrist-worn sensors. In International Conference on Tangible and Embedded Interaction (TEI 2010), pages 57--64, Cambridge MA, USA, 01/2010 2010. ACM Press, ACM Press.

Digital Library

[4]

A. Feldman, E. M. Tapia, S. Sadi, P. Maes, and C. Schmandt. Reachmedia: On-the-move interaction with everyday objects. In Proceedings of the Ninth IEEE International Symposium on Wearable Computers, ISWC '05, pages 52--59, Washington, DC, USA, 2005. IEEE Computer Society.

Digital Library

[5]

K. P. Fishkin, M. Philipose, and A. D. Rea. Hands-on RFID: wireless wearables for detecting use of objects. In Ninth IEEE International Symposium on Wearable Computers (ISWC 2005), 18-21 October 2005, Osaka, Japan, pages 38--43. IEEE Computer Society, 2005.

Digital Library

[6]

N. Gillian and J. A. Paradiso. The gesture recognition toolkit. The Journal of Machine Learning Research, 15(1):3483--3487, 2014.

Digital Library

[7]

T. Grosse-Puppendahl. Capacitive Sensing and Communication for Ubiquitous Interaction and Env. Perception. PhD thesis, TU Darmstadt, May 2015.

[8]

T. Grosse-Puppendahl, E. Berlin, and M. Borazio. Enhancing accelerometer-based activity recognition with capacitive proximity sensing. In Ambient Intelligence, volume 7683 of LNCS, pages 17--32. Springer, 2012.

[9]

A. Hein and T. Kirste. A hybrid approach for recognizing adls and care activities using inertial sensors and rfid. In C. Stephanidis, editor, Intelligent and Ubiquitous Interaction Environments, 5th International Conference, UAHCI 2009, Lecture Notes in Computer Science, pages 178--188, San Diego, CA, USA, 2009. Springer.

Digital Library

[10]

A. Latwesen and P. E. Patterson. Identification of lower arm motions using the emg signals of shoulder muscles. Medical Engineering and Physics, 16(2), 1994.

[11]

T. Maekawa, Y. Kishino, Y. Yanagisawa, and Y. Sakurai. Wristsense: Wrist-worn sensor device with camera for daily activity recognition. In Pervasive Computing and Communications Workshops, 2012 IEEE Intl. Conf. on, pages 510--512, March 2012.

[12]

T. Maekawa, Y. Yanagisawa, Y. Kishino, K. Ishiguro, K. Kamei, Y. Sakurai, and T. Okadome. Object-based activity recognition with heterogeneous sensors on wrist. In Pervasive Computing, volume 6030 of Lecture Notes in Computer Science, pages 246--264. Springer Berlin, 2010.

Digital Library

[13]

D. J. Patterson, D. Fox, H. A. Kautz, and M. Philipose. Fine-grained activity recognition by aggregating abstract object usage. In Ninth IEEE International Symposium on Wearable Computers (ISWC 2005), 18-21 October 2005, Osaka, Japan, pages 44--51. IEEE Computer Society, 2005.

Digital Library

[14]

M. Philipose, K. P. Fishkin, M. Perkowitz, D. J. Patterson, D. Fox, H. Kautz, and D. Hahnel. Inferring activities from interactions with objects. Pervasive Computing, 3(4):50âĂŞ57, 2004.

Digital Library

[15]

J. Rekimoto. Gesturewrist and gesturepad: Unobtrusive wearable interaction devices. In Proceedings of the 5th IEEE International Symposium on Wearable Computers, ISWC '01, pages 21--, Washington, DC, USA, 2001. IEEE Computer Society.

Digital Library

[16]

A. Schmidt, H. Gellersen, and C. Merz. Enabling implicit human computer interaction: A wearable rfid-tag reader. In Fourth International Symposium on Wearable Computers (ISWC 2000), Atlanta, Georgia, USA, 18--21 October 2000, Proceedings., pages 193--194, 2000.

Digital Library

[17]

F. Tavakolizadeh, J. Gu, and B. Saket. Traceband: Locating missing items by visual remembrance. In Proceedings of the Adjunct Publication of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST'14 Adjunct, pages 109--110, New York, NY, USA, 2014. ACM.

Digital Library

[18]

R. Want. An introduction to rfid technology. IEEE Pervasive Computing, 5(1):25--33, Jan. 2006.

Digital Library

[19]

K. Wolf and J. Willaredt. Pickring: seamless interaction through pick-up detection. In Proceedings of the 6th Augmented Human International Conference, AH 2015, Singapore, March 9--11, 2015, pages 13--20, 2015.

Digital Library

[20]

A. Zinnen, K. Van Laerhoven, and B. Schiele. Toward recognition of short and non-repetitive activities from wearable sensors. In Proceedings of the 2007 European conference on Ambient intelligence, pages 142--158. Springer-Verlag, 2007.

Digital Library

Cited By

Reed CSkach SStrohmeier PMcPherson A(2022)Singing Knit: Soft Knit Biosensing for Augmenting Vocal PerformancesProceedings of the Augmented Humans International Conference 202210.1145/3519391.3519412(170-183)Online publication date: 13-Mar-2022
https://dl.acm.org/doi/10.1145/3519391.3519412
Reed C(2022)Examining Embodied Sensation and Perception in SingingProceedings of the Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3490149.3503581(1-7)Online publication date: 13-Feb-2022
https://dl.acm.org/doi/10.1145/3490149.3503581
Reed CMcPherson AKaltenbrunner MMurer MWolf KOakley I(2021)Surface Electromyography for Sensing Performance Intention and Musical Imagery in VocalistsProceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3430524.3440641(1-11)Online publication date: 14-Feb-2021
https://dl.acm.org/doi/10.1145/3430524.3440641
Show More Cited By

Recommendations

GaitTracker: 3D Skeletal Tracking for Gait Analysis Based on Inertial Measurement Units
Gait rehabilitation is a common method of postoperative recovery after the user sustains an injury or disability. However, traditional gait rehabilitations are usually performed under the supervision of rehabilitation specialists, which implies that the ...
Gesture recognition for transhumeral prosthesis control using EMG and NIR

A key challenge associated with myoelectric prosthesis limbs is the acquisition of a good quality gesture intent signal from the residual anatomy of an amputee. In this study, the authors aim to overcome this limitation by observing the classification ...
A low cost wearable wireless sensing system for paretic hand management after stroke

This paper describes the design of a low cost wearable hand exercise device that can assist repetitive wrist and finger exercise for stroke patients. The design of this device was guided by neurobiological principles of motor learning, such as sensory-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

AH '16: Proceedings of the 7th Augmented Human International Conference 2016

February 2016

258 pages

ISBN:9781450336802

DOI:10.1145/2875194

General Chairs:
Jean-Marc Seigneur
University of Geneva, Switzerland
,
Jose M. Hernandez-Munoz
Telefonica I+D, Spain
,
Paul McCullagh
University of Ulster, United Kingdom
,
Program Chairs:
Albrecht Schmidt
University of Stuttgart, Germany
,
Tsutomu Terada
University of Kobe, Japan
,
Woontack Woo
Korea Advanced Institute of Science and Technology
,
Pranav Mistry
Samsung Research America, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

University of Geneva

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 February 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AH '16

AH '16: Augmented Human International Conference 2016

February 25 - 27, 2016

Geneva, Switzerland

Acceptance Rates

AH '16 Paper Acceptance Rate 21 of 138 submissions, 15%;

Overall Acceptance Rate 121 of 306 submissions, 40%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
379
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)1

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Reed CSkach SStrohmeier PMcPherson A(2022)Singing Knit: Soft Knit Biosensing for Augmenting Vocal PerformancesProceedings of the Augmented Humans International Conference 202210.1145/3519391.3519412(170-183)Online publication date: 13-Mar-2022
https://dl.acm.org/doi/10.1145/3519391.3519412
Reed C(2022)Examining Embodied Sensation and Perception in SingingProceedings of the Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3490149.3503581(1-7)Online publication date: 13-Feb-2022
https://dl.acm.org/doi/10.1145/3490149.3503581
Reed CMcPherson AKaltenbrunner MMurer MWolf KOakley I(2021)Surface Electromyography for Sensing Performance Intention and Musical Imagery in VocalistsProceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3430524.3440641(1-11)Online publication date: 14-Feb-2021
https://dl.acm.org/doi/10.1145/3430524.3440641
Bowman MZhang X(2021)Dynamic Pre-Grasp Planning when Tracing a Moving Object Through a Multi-Agent Perspective2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS51168.2021.9636666(9408-9414)Online publication date: 27-Sep-2021
https://doi.org/10.1109/IROS51168.2021.9636666
Karolus JKilian AKosch TSchmidt AWozniak PWakkary RAndersen KOdom WDesjardins APetersen M(2020)Hit the Thumb Jack! Using Electromyography to Augment the Piano KeyboardProceedings of the 2020 ACM Designing Interactive Systems Conference10.1145/3357236.3395500(429-440)Online publication date: 3-Jul-2020
https://dl.acm.org/doi/10.1145/3357236.3395500
Kwon YShatilov KLee LKumyol SLam KYau YHui P(2020)MyoKey: Surface Electromyography and Inertial Motion Sensing-based Text Entry in AR2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)10.1109/PerComWorkshops48775.2020.9156084(1-4)Online publication date: Mar-2020
https://doi.org/10.1109/PerComWorkshops48775.2020.9156084
Miozzi CErrico VMarrocco GGruppioni ESaggio G(2020)RFID interface for compact pliable EMG wireless epidermal sensor2020 IEEE International Workshop on Metrology for Industry 4.0 & IoT10.1109/MetroInd4.0IoT48571.2020.9138182(202-206)Online publication date: Jun-2020
https://doi.org/10.1109/MetroInd4.0IoT48571.2020.9138182
Fan JFan XTian FLi YLiu ZSun WWang H(2018)What is That in Your Hand?Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/32870392:4(1-24)Online publication date: 27-Dec-2018
https://dl.acm.org/doi/10.1145/3287039
Hauser NWade E(2018)Detecting Reach to Grasp Activities using Motion and Muscle Activation Data2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)10.1109/EMBC.2018.8512937(3264-3267)Online publication date: Jul-2018
https://doi.org/10.1109/EMBC.2018.8512937
Xiao ZMenon C(2017)Counting Grasping Action Using Force Myography: An Exploratory Study With Healthy IndividualsJMIR Rehabilitation and Assistive Technologies10.2196/rehab.69014:1(e5)Online publication date: 16-May-2017
https://doi.org/10.2196/rehab.6901

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten