poster

Hand and foot gesture interaction for handheld devices

Authors:

Zhihan Lu,

Muhammad Sikandar Lal Khan,

Shafiq Ur RéhmanAuthors Info & Claims

MM '13: Proceedings of the 21st ACM international conference on Multimedia

Pages 621 - 624

https://doi.org/10.1145/2502081.2502163

Published: 21 October 2013 Publication History

Get Access

Abstract

In this paper we present hand and foot based immersive multimodal interaction approach for handheld devices. A smart phone based immersive football game is designed as a proof of concept. Our proposed method combines input modalities (i.e. hand & foot) and provides a coordinated output to both modalities along with audio and video. In this work, human foot gesture is detected and tracked using template matching method and Tracking-Learning-Detection (TLD) framework. We evaluated our system's usability through a user study in which we asked participants to evaluate proposed interaction method. Our preliminary evaluation demonstrates the efficiency and ease of use of proposed multimodal interaction approach.

Supplementary Material

suppl.mov (mm082s.avi)

Supplemental video

Download
6.01 MB

References

[1]

T. Augsten, K. Kaefer, R. Meusel, C. Fetzer, D. Kanitz, T. Stoff, T. Becker, C. Holz, and P. Baudisch. Multitoe: high-precision interaction with back-projected floors based on high-resolution multi-touch input. In 23rd Annual ACM Sym. UIST, 2010.

Digital Library

Google Scholar

[2]

A. Crossan, S. Brewster, and A. Ng. Foot Tapping for Mobile Interaction. In 24th BCS Conf. HCI, UK, 2010.

Digital Library

Google Scholar

[3]

K. Grauman, M. Betke, J. Gips, and G. R. Bradski. Communication Via Eye Blinks - Detection and Duration Analysis in Real Time. In IEEE Conf. Comp. V. and P. Recognition, 2001.

Google Scholar

[4]

T. Han, J. Alexander, A. Karnik, P. Irani, and S. Subramanian. Kick: Investigating the Use of Kick Gestures for Mobile Interactions. In 13th Int. Conf. HCI with Mobile Devices and Services, 2011.

Digital Library

Google Scholar

[5]

A. Jaimes and N. Sebe. Multimodal Human computer Interaction: A survey. Comp. V. and Im. Understanding, 108:116--134, 2007.

Digital Library

Google Scholar

[6]

Z. Kalal, K. Mikolajczyk, and J. Matas. Tracking Learning Detection. IEEE Trans. on Pattern A. and M. Int.(TPAMI), 6(1), 2010.

Digital Library

Google Scholar

[7]

P. Mistry and P. Maes. SixthSense: a wearable gestural interface. In ACM SIGGRAPH ASIA 2009 Sketches, 2009.

Digital Library

Google Scholar

[8]

S. Réhman, A. Khan, and H. Li. Interactive Feet for Mobile Immersive Interaction. In ACM Int. Workshop MobiVis Workshop at MobileHCI, 2012.

Google Scholar

[9]

N. Sangsuriyachot and M. Sugimoto. Novel Interaction Techniques based on a Combination of Hand and Foot Gestures in Tabletop Environments. In 10th Asia Pacific Conf. HCI, 2012.

Digital Library

Google Scholar

[10]

J.-H. Seo, J.-Y. Yang, and D.-S. Kwon. Laser scanner based Foot Motion Detection for Intuitive Robot User Interface System. In IEEE RO-MAN, 2012.

Crossref

Google Scholar

[11]

T. Simpson, M. Gauthier, and A. Prochazka. Evaluation of Tooth-Click triggering and Speech Recognition in Assistive Technology for Computer Access. Neurorehabil Neural Repair, 24:188--194, 2010.

Crossref

Google Scholar

[12]

D. Valkov, F. Steinicke, G. Bruder, and K. Hinrichs. Traveling in 3D Virtual Environments with Foot Gestures and a Multi-Touch enabled WIM. In Int. Conf. Virtual Reality, 2010.

Google Scholar

[13]

D. Watson and L. A. Clark. Development and Validation of Brief Measures of Positive and Negative Affect: The PANAS scales. J. Personality and Social Psychology, 54(6):1063--1070, 1988.

Crossref

Google Scholar

Cited By

View all

Shen JZhou XWu WWang LChen Z(2023)Worldwide Overview and Country Differences in Metaverse Research: A Bibliometric AnalysisSustainability10.3390/su1504354115:4(3541)Online publication date: 15-Feb-2023
https://doi.org/10.3390/su15043541
Li XMatthies D(2022)ShoeTect: Detecting Body Posture, Ambulation Activity, Gait Abnormalities, and Terrain with Multisensory Smart FootwearProceedings of the 7th International Workshop on Sensor-based Activity Recognition and Artificial Intelligence10.1145/3558884.3558904(1-10)Online publication date: 19-Sep-2022
https://dl.acm.org/doi/10.1145/3558884.3558904
Xu PLiu HWu L(2020)Extreme Learning Machines Classification of Kick GestureProceedings of ELM201910.1007/978-3-030-58989-9_18(171-180)Online publication date: 12-Sep-2020
https://doi.org/10.1007/978-3-030-58989-9_18
Show More Cited By

Index Terms

Hand and foot gesture interaction for handheld devices
1. Information systems

Recommendations

Multimodal Hand and Foot Gesture Interaction for Handheld Devices
Special Issue on Multiple Sensorial (MulSeMedia) Multimodal Media : Advances and Applications

We present a hand-and-foot-based multimodal interaction approach for handheld devices. Our method combines input modalities (i.e., hand and foot) and provides a coordinated output to both modalities along with audio and video. Human foot gesture is ...
Novel interaction techniques based on a combination of hand and foot gestures in tabletop environments
APCHI '12: Proceedings of the 10th asia pacific conference on Computer human interaction

Interactive tables, or tabletop devices, employ multi-finger gestures to interact with digital contents on a table's surface. Many studies have confirmed the convenience and intuitiveness of multi-finger gestures performed with the hands. However, there ...
Context-based conversational hand gesture classification in narrative interaction
ICMI '13: Proceedings of the 15th ACM on International conference on multimodal interaction

Communicative hand gestures play important roles in face-to-face conversations. These gestures are arbitrarily used depending on an individual; even when two speakers narrate the same story, they do not always use the same hand gesture (movement, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

MM '13: Proceedings of the 21st ACM international conference on Multimedia

October 2013

1166 pages

ISBN:9781450324045

DOI:10.1145/2502081

General Chairs:
Alejandro (Alex) Jaimes
Yahoo!, Spain
,
Nicu Sebe
University of Trento, Italy
,
Nozha Boujemaa
INRIA, France
,
Program Chairs:
Daniel Gatica-Perez
IDIAP & EPFL, Switzerland
,
David A. Shamma
Yahoo!, USA
,
Marcel Worring
University of Amsterdam, The Netherlands
,
Roger Zimmermann
National University of Singapore, Singapore

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

MM '13

Sponsor:

SIGMM

MM '13: ACM Multimedia Conference

October 21 - 25, 2013

Barcelona, Spain

Acceptance Rates

MM '13 Paper Acceptance Rate 47 of 235 submissions, 20%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
462
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Shen JZhou XWu WWang LChen Z(2023)Worldwide Overview and Country Differences in Metaverse Research: A Bibliometric AnalysisSustainability10.3390/su1504354115:4(3541)Online publication date: 15-Feb-2023
https://doi.org/10.3390/su15043541
Li XMatthies D(2022)ShoeTect: Detecting Body Posture, Ambulation Activity, Gait Abnormalities, and Terrain with Multisensory Smart FootwearProceedings of the 7th International Workshop on Sensor-based Activity Recognition and Artificial Intelligence10.1145/3558884.3558904(1-10)Online publication date: 19-Sep-2022
https://dl.acm.org/doi/10.1145/3558884.3558904
Xu PLiu HWu L(2020)Extreme Learning Machines Classification of Kick GestureProceedings of ELM201910.1007/978-3-030-58989-9_18(171-180)Online publication date: 12-Sep-2020
https://doi.org/10.1007/978-3-030-58989-9_18
Sun YArmengol-Urpi AReddy Kantareddy SSiegel JSarma S(2019)MagicHand: Interact with IoT Devices in Augmented Reality Environment2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)10.1109/VR.2019.8798053(1738-1743)Online publication date: Mar-2019
https://doi.org/10.1109/VR.2019.8798053
Shi XPan JHu ZLin JGuo SLiao MPan YLiu L(2019)Accurate and Fast Classification of Foot Gestures for Virtual Locomotion2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR.2019.000-6(178-189)Online publication date: Oct-2019
https://doi.org/10.1109/ISMAR.2019.000-6
Venkatnarayan RShahzad M(2018)Gesture Recognition Using Ambient LightProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/31917722:1(1-28)Online publication date: 26-Mar-2018
https://dl.acm.org/doi/10.1145/3191772
Felberbaum YLanir JMandryk RHancock MPerry MCox A(2018)Better Understanding of Foot GesturesProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3173908(1-12)Online publication date: 21-Apr-2018
https://dl.acm.org/doi/10.1145/3173574.3173908
Liu YWang XYan K(2018)Hand gesture recognition based on concentric circular scan lines and weighted K-nearest neighbor algorithmMultimedia Tools and Applications10.1007/s11042-016-4265-677:1(209-223)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1007/s11042-016-4265-6
Matthies DRoumen TKuijper AUrban BJones MTscheligi MRogers YMurray-Smith R(2017)CapSolesProceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3098279.3098545(1-14)Online publication date: 4-Sep-2017
https://dl.acm.org/doi/10.1145/3098279.3098545
Kim YAhn EKim G(2017)Evaluation of hand-foot coordinated quadruped interaction for mobile applicationsJournal on Multimodal User Interfaces10.1007/s12193-017-0254-y11:4(315-325)Online publication date: 16-Nov-2017
https://doi.org/10.1007/s12193-017-0254-y
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Multimodal Hand and Foot Gesture Interaction for Handheld Devices

Novel interaction techniques based on a combination of hand and foot gestures in tabletop environments

Context-based conversational hand gesture classification in narrative interaction