research-article

WiFinger: talk to your smart devices with finger-grained gesture

Authors:

Liusheng HuangAuthors Info & Claims

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

Pages 250 - 261

https://doi.org/10.1145/2971648.2971738

Published: 12 September 2016 Publication History

Abstract

In recent literatures, WiFi signals have been widely used to "sense" people's locations and activities. Researchers have exploited the characteristics of wireless signals to "hear" people's talk and "see" keystrokes by human users. Inspired by the excellent work of relevant scholars, we turn to explore the field of human-computer interaction using finger-grained gestures under WiFi environment. In this paper, we present Wi-Finger - the first solution using ubiquitous wireless signals to achieve number text input in WiFi devices. We implement a prototype of WiFinger on a commercial Wi-Fi infrastructure. Our scheme is based on the key intuition that while performing a certain gesture, the fingers of a user move in a unique formation and direction and thus generate a unique pattern in the time series of Channel State Information (CSI) values. WiFinger is deigned to recognize a set of finger-grained gestures, which are further used to realize continuous text input in off-the-shelf WiFi devices. As the results show, WiFinger achieves up to 90.4% average classification accuracy for recognizing 9 digits finger-grained gestures from American Sign Language (ASL), and its average accuracy for single individual number text input in desktop reaches 82.67% within 90 digits.

Supplementary Material

ZIP File (p250-li.zip)

Supplemental material.

Download
43.62 MB

References

[1]

Ali, K., Liu, A. X., Wang, W., and Shahzad, M. Keystroke recognition using wifi signals. In Proc of ACM MobiCom (2015), 90--102.

Digital Library

[2]

Wang, G., Zou, Y., Zhou, Z., k. wu, and Ni, L. We can hear you with wi-fi! In Proc of ACM MobiCom (2014), 593--604.

Digital Library

[3]

Gupta, S., Morris, D., Patel, S., and Tan, D. Soundwave: using the doppler effect to sense gestures. In Proc of the SIGCHI Conference on Human Factors in Computing Systems (2012), 1911--1914.

Digital Library

[4]

Leap Motion. https://www.leapmotion.com.

[5]

Microsoft Kinect. http://www.roborealm.com/help/MicrosoftKinect.php.

[6]

Pu, Q., Gupta, S., Gollakota, S., and Patel, S. Whole-home gesture recognition using wireless signals. In Proc of ACM MobiCom (2013), 27--38.

Digital Library

[7]

Adib, F., and Katabi, D. See through walls with wifi! In Proc of ACM SIGCOMM (2013), 75--86.

Digital Library

[8]

Melgarejo, P., Zhang, X., Ramanathan, P., and Chu, D. Leveraging directional antenna capabilities for fine-grained gesture recognition. In Proc of ACM UbiComp (2014), 541--551.

Digital Library

[9]

Halperin, D., Hu, W., Sheth, A., and Wetherall, D. Tool release: Gathering 802.11n traces with channel state information. ACM SIGCOMM Computer Communication Review 41, 1 (2011), 53--53.

Digital Library

[10]

Sturman, D. J., and Zeltzer, D. A survey of glove-based input. IEEE Computer Graphics and Applications 14, 1 (1994), 30--39.

Digital Library

[11]

Dipietro, L., Sabatini, A. M., and Dario, P. A survey of glove-based systems and their applications. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 38, 4 (2008), 461--482.

Digital Library

[12]

Finger Gesture. http://www.lifeprint.com/dictionary.htm.

[13]

Scholz, M., Riedel, T., Hock, M., and Beigl, M. Device-free and device-bound activity recognition using radio signal strength. In Proc of the 4th Augmented Human International Conference, ACM (2013), 100--107.

Digital Library

[14]

Tarzia, S. P., Dick, R. P., Dinda, P. A., and Memik, G. Sonar-based measurement of user presence and attention. In Proc of ACM UbiComp (2009), 89--92.

Digital Library

[15]

Zhao, Y., Patwari, N., Phillips, J. M., and Venkatasubramanian, S. Radio tomographic imaging and tracking of stationary and moving people via kernel distance. In Proc of the 12th international conference on Information processing in sensor networks (2013), 229--240.

Digital Library

[16]

Starner, T., and Pentland, A. Real-time american sign language recognition from video using hidden markov models. In Motion-Based Recognition. Springer, 1997, 227--243.

[17]

Juan, P. W., Kölsch, M., Stern, H., and Edan, Y. Vision-based hand-gesture applications. Communications of the ACM 54, 2 (2011), 60--71.

Digital Library

[18]

Rehg, J. M., and Kanade, T. Visual tracking of high dof articulated structures: an application to human hand tracking. In Computer Vision ECCV'94. Springer, 1994, 35--46.

Digital Library

[19]

Ketabdar, H., Roshandel, M., and Kamer, A. Y. Towards using embedded magnetic field sensor for around mobile device 3d interaction. In Proc of the 12th international conference on Human computer interaction with mobile devices and services (2010), 153--156.

Digital Library

[20]

Ketabdar, H., Moghadam, P., Naderi, B., and Roshandel, M. Magnetic signatures in air for mobile devices. In Proc of the 14th international conference on Human-computer interaction with mobile devices and services companion (2012), 185--188.

Digital Library

[21]

Agrawal, S., Constandache, I., Gaonkar, S., Roy, C. R., Caves, K., and DeRuyter, F. Using mobile phones to write in air. In Proc of ACM Mobisys (2011), 15--28.

Digital Library

[22]

Park, T., Lee, J., Hwang, I., Yoo, C., Nachman, L., and Song, J. E-gesture: a collaborative architecture for energy-efficient gesture recognition with hand-worn sensor and mobile devices. In Proc of ACM SenSys (2011), 260--273.

Digital Library

[23]

Sigg, S., Shi, S., Buesching, F., Ji, Y., and Wolf, L. Leveraging rf-channel fluctuation for activity recognition: Active and passive systems, continuous and rssi-based signal features. In Proc of International Conference on Advances in Mobile Computing & Multimedia (2013), 43.

Digital Library

[24]

Sigg, S., Scholz, M., Shi, S., Ji, Y., and Beigl, M. Rf-sensing of activities from non-cooperative subjects in device-free recognition systems using ambient and local signals. Mobile Computing, IEEE Transactions on 13, 4 (2014), 907--920.

Digital Library

[25]

Abdelnasser, H., Youssef, M., and Harras, K. A. Wigest: A ubiquitous wifi-based gesture recognition system. In Computer Communications (INFOCOM), 2015 IEEE Conference on (2015), 1472--1480.

[26]

Han, C., Wu, K., Wang, Y., and Ni, L. M. Wifall: Device-free fall detection by wireless networks. In Proc of IEEE INFOCOM (2014), 271--279.

[27]

Wang, Y., Liu, J., Chen, Y., Gruteser, M., Yang, J., and Liu, H. E-eyes: device-free location-oriented activity identification using fine-grained wifi signatures. In Proc of ACM MobiCom (2014), 617--628.

Digital Library

[28]

Zhou, Z., Yang, Z., Wu, C., Shangguan, L., and Liu, Y. Towards omnidirectional passive human detection. In Proc of IEEE INFOCOM (2013), 3057--3065.

[29]

Xi, W., Zhao, J., Li, X.-Y., Zhao, K., Tang, S., Liu, X., and Jiang, Z. Electronic frog eye: Counting crowd using wifi. In Proc of IEEE INFOCOM (2014), 361--369.

[30]

Nandakumar, R., Kellogg, B., and Gollakota, S. Wi-fi gesture recognition on existing devices. CoRR abs/1411.5394 (2014).

[31]

Sen, S., Lee, J., Kim, K.-H., and Congdon, P. Avoiding multipath to revive inbuilding wifi localization. In Proc of the 11th annual international conference on Mobile systems, applications, and services (2013), 249--262.

Digital Library

[32]

Xiao, J., Wu, K., Yi, Y., and Ni, L. M. Fifs: Fine-grained indoor fingerprinting system. In Computer Communications and Networks (ICCCN), 2012 21st International Conference on (2012), 1--7.

[33]

Yang, Z., Zhou, Z., and Liu, Y. From rssi to csi: Indoor localization via channel response. ACM Computing Surveys (CSUR) 46, 2 (2013), 25.

Digital Library

[34]

Kellogg, B., Talla, V., and Gollakota, S. Bringing gesture recognition to all devices. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14) (2014), 303--316.

Digital Library

[35]

Lyonnet, B., Ioana, C., and Amin, M. G. Human gait classification using microdoppler time-frequency signal representations. In Radar Conference, 2010 IEEE (2010), 915--919.

[36]

Adib, F., Kabelac, Z., Katabi, D., and Miller, R. C. 3d tracking via body radio reflections. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14) (2014), 317--329.

Digital Library

[37]

Chen, B., Yenamandra, V., and Srinivasan, K. Tracking keystrokes using wireless signals. In Proc of the 13th Annual International Conference on Mobile Systems, Applications, and Services (2015), 31--44.

Digital Library

[38]

Halperin, D., Hu, W., Sheth, A., and Wetherall, D. Two antennas are better than one: A measurement study of 802.11 n.

[39]

Davies, L., and Gather, U. The identification of multiple outliers. Journal of the American Statistical Association 88, 423 (1993), 782--792.

[40]

Wang, W., Liu, A. X., Shahzad, M., Ling, K., and Lu, S. Understanding and modeling of wifi signal based human activity recognition. In Proc of ACM MobiCom (2015), 65--76.

Digital Library

[41]

Mallat, S. G. A theory for multiresolution signal decomposition: the wavelet representation. Pattern Analysis and Machine Intelligence, IEEE Transactions on 11, 7 (1989), 674--693.

Digital Library

[42]

Müller, M. Dynamic time warping. Information retrieval for music and motion (2007), 69--84.

Digital Library

[43]

Halperin, D., Hu, W., Sheth, A., and Wetherall, D. Predictable 802.11 packet delivery from wireless channel measurements. In Proc of ACM SIGCOMM (2010), 159--170.

Digital Library

[44]

Committee, I. C. S. L. M. S., et al. Wireless LAN medium access control (MAC) and physical layer (PHY) specifications. IEEE Std, 1997.

[45]

Perahia, E., and Stacey, R. Next Generation Wireless LANS: 802.11 n and 802.11 ac. Cambridge university press, 2013.

Digital Library

[46]

Quegan, S. Spotlight synthetic aperture radar: Signal processing algorithms. International Journal for Numerical Methods in Engineering 83, 11 (2010), 1498--1517.

[47]

Baldauf, M., Dustdar, S., and Rosenberg, F. A survey on context-aware systems. International Journal of Ad Hoc and Ubiquitous Computing 2, 4 (2007), 263--277.

Digital Library

Cited By

Li SLiu ZLv QZou YZhang YZhang D(2025)WiLife: Long-Term Daily Status Monitoring and Habit Mining of the Elderly Leveraging Ubiquitous Wi-Fi SignalsACM Transactions on Computing for Healthcare10.1145/36893736:1(1-29)Online publication date: 23-Jan-2025
https://dl.acm.org/doi/10.1145/3689373
Xu MHan YZhu NYang B(2025)A method for recognizing individual dynamic thermal adaptations using wireless signalsEnergy and Buildings10.1016/j.enbuild.2025.115448333(115448)Online publication date: Apr-2025
https://doi.org/10.1016/j.enbuild.2025.115448
Ge FDai ZYang ZWu FTan L(2024)VBCNet: A Hybird Network for Human Activity RecognitionSensors10.3390/s2423779324:23(7793)Online publication date: 5-Dec-2024
https://doi.org/10.3390/s24237793
Show More Cited By

Index Terms

WiFinger: talk to your smart devices with finger-grained gesture
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

SignFi: Sign Language Recognition Using WiFi

We propose SignFi to recognize sign language gestures using WiFi. SignFi uses Channel State Information (CSI) measured by WiFi packets as the input and a Convolutional Neural Network (CNN) as the classification algorithm. Existing WiFi-based sign gesture ...
WiFinger: leveraging commodity WiFi for fine-grained finger gesture recognition
MobiHoc '16: Proceedings of the 17th ACM International Symposium on Mobile Ad Hoc Networking and Computing

Gesture recognition has become increasingly important in human-computer interaction (HCI) and can support a broad array of emerging applications, such as smart home, virtual reality, and mobile gaming. Traditional approaches usually rely on dedicated ...
WiDraw: Enabling Hands-free Drawing in the Air on Commodity WiFi Devices
MobiCom '15: Proceedings of the 21st Annual International Conference on Mobile Computing and Networking

This paper demonstrates that it is possible to leverage WiFi signals from commodity mobile devices to enable hands-free drawing in the air. While prior solutions require the user to hold a wireless transmitter, or require custom wireless hardware, or ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 2016

1288 pages

ISBN:9781450344616

DOI:10.1145/2971648

General Chairs:
Paul Lukowicz
DFKI and University of Kaiserslautern
,
Antonio Krüger
DFKI and Saarland University
,
Program Chairs:
Andreas Bulling
Max Planck Institute for Informatics
,
Youn-Kyung Lim
KAIST
,
Shwetak N. Patel
University of Washington

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 September 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Best Paper

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Science Foundation of Jiangsu Province of China
National Natural Science Foundation of China

Conference

UbiComp '16

Sponsor:

UbiComp '16: The 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 12 - 16, 2016

Germany, Heidelberg

Acceptance Rates

UbiComp '16 Paper Acceptance Rate 101 of 389 submissions, 26%;

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

219
Total Citations
View Citations
3,189
Total Downloads

Downloads (Last 12 months)106
Downloads (Last 6 weeks)8

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li SLiu ZLv QZou YZhang YZhang D(2025)WiLife: Long-Term Daily Status Monitoring and Habit Mining of the Elderly Leveraging Ubiquitous Wi-Fi SignalsACM Transactions on Computing for Healthcare10.1145/36893736:1(1-29)Online publication date: 23-Jan-2025
https://dl.acm.org/doi/10.1145/3689373
Xu MHan YZhu NYang B(2025)A method for recognizing individual dynamic thermal adaptations using wireless signalsEnergy and Buildings10.1016/j.enbuild.2025.115448333(115448)Online publication date: Apr-2025
https://doi.org/10.1016/j.enbuild.2025.115448
Ge FDai ZYang ZWu FTan L(2024)VBCNet: A Hybird Network for Human Activity RecognitionSensors10.3390/s2423779324:23(7793)Online publication date: 5-Dec-2024
https://doi.org/10.3390/s24237793
Wang ZGeng CSun ZChen ZLiu SGuo BYu Z(2024)Size Matters: Characterizing the Effect of Target Size on Wi-Fi Sensing Based on the Fresnel Zone ModelProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997268:4(1-22)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699726
Wang ZChen ZZhang YGeng CSong WSun ZGuo BYu ZChen L(2024)GrainSense: A Wireless Grain Moisture Sensing System Based on Wi-Fi SignalsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785898:3(1-25)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678589
Sheng BHan RXiao FGuo ZGui L(2024)MetaFormerProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435508:1(1-27)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643550
Cominelli MShahcheraghi SLink JHollick MCerutti FGringoli FAsadi A(2024)Physical-Layer Privacy via Randomized Beamforming Against Adversarial Wi-Fi Sensing: Analysis, Implementation, and EvaluationIEEE Transactions on Wireless Communications10.1109/TWC.2024.348547723:12(19603-19617)Online publication date: Dec-2024
https://doi.org/10.1109/TWC.2024.3485477
Mao YGuo ZSheng BGui LXiao F(2024)Wi-Cro: WiFi-Based Cross Domain Activity Recognition via Modified GANIEEE Transactions on Vehicular Technology10.1109/TVT.2024.340445273:10(14961-14973)Online publication date: Oct-2024
https://doi.org/10.1109/TVT.2024.3404452
Zheng XYang KXiong JLiu LMa H(2024)Pushing the Limits of WiFi Sensing With Low Transmission RatesIEEE Transactions on Mobile Computing10.1109/TMC.2024.337404623:11(10265-10279)Online publication date: Nov-2024
https://doi.org/10.1109/TMC.2024.3374046
Sheng BHan RCai HXiao FGui LGuo Z(2024)CDFi: Cross-Domain Action Recognition Using WiFi SignalsIEEE Transactions on Mobile Computing10.1109/TMC.2023.334893923:8(8463-8477)Online publication date: Aug-2024
https://doi.org/10.1109/TMC.2023.3348939
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten