research-article

We can hear you with Wi-Fi!

Authors:

Lionel M. NiAuthors Info & Claims

MobiCom '14: Proceedings of the 20th annual international conference on Mobile computing and networking

Pages 593 - 604

https://doi.org/10.1145/2639108.2639112

Published: 07 September 2014 Publication History

Abstract

Recent literature advances Wi-Fi signals to "see" people's motions and locations. This paper asks the following question: Can Wi-Fi "hear" our talks? We present WiHear, which enables Wi-Fi signals to "hear" our talks without deploying any devices. To achieve this, WiHear needs to detect and analyze fine-grained radio reflections from mouth movements. WiHear solves this micro-movement detection problem by introducing Mouth Motion Profile that leverages partial multipath effects and wavelet packet transformation. Since Wi-Fi signals do not require line-of-sight, WiHear can "hear" people talks within the radio range. Further, WiHear can simultaneously "hear" multiple people's talks leveraging MIMO technology. We implement WiHear on both USRP N210 platform and commercial Wi-Fi infrastructure. Results show that within our pre-defined vocabulary, WiHear can achieve detection accuracy of 91% on average for single individual speaking no more than 6 words and up to 74% for no more than 3 people talking simultaneously. Moreover, the detection accuracy can be further improved by deploying multiple receivers from different angles.

References

[1]

Leap Motion, https://www.leapmotion.com/.

[2]

Xbox Kinect, http://www.xbox.com/en-US/kinect.

[3]

GNU software defined radio, http://www.gnu.org/software/gnuradio.

[4]

TP-LINK 2.4GHz 24dBi Grid Parabolic Antenna, http://www.tp-link.com/en/products/details/?categoryid=2474&model=TL-ANT2424B#over.

[5]

Oscilloquartz SA, OSA 5200B GPS Clock. http://www.oscilloquartz.com.

[6]

Universal Software Radio Peripheral. Ettus Research LLC, http://www.ettus.com.

[7]

Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications. IEEE Std 802.11, 2012.

[8]

F. Adib, Z. Kabelac, D. Katabi, and R. C. Miller. 3d tracking via body radio reflections. In USENIX NSDI, 2014.

Digital Library

[9]

F. Adib and D. Katabi. See through walls with wi-fi! In ACM SIGCOMM, 2013.

Digital Library

[10]

S. Agrawal, I. Constandache, S. Gaonkar, R. R. Choudhury, K. Cave, and F. DeRuyter. Using mobile phones to write in air. In ACM MobiSys, 2011.

Digital Library

[11]

M. Aynehband, A. M. Rahmani, and S. Setayeshi. Coast: Context-aware pervasive speech recognition system. In IEEE International Symposium on Wireless and Pervasive Computing, 2011.

[12]

D. Bharadia, K. R. Joshi, and S. Katti. Full duplex backscatter. In ACM HotNets, 2013.

Digital Library

[13]

C. Bo, X. Jian, X.-Y. Li, X. Mao, Y. Wang, and F. Li. You're Driving and Texting: Detecting Drivers Using Personal Smart Phones by Leveraging Inertial Sensors. In ACM MobiCom, 2013.

Digital Library

[14]

J. Bradbury. Linear predictive coding. Mc G. Hill.

[15]

G. L. Charvat, L. C. Kempel, E. J. Rothwell, C. M. Coleman, and E. L. Mokole. A through-dielectric radar imaging system. IEEE Transactions on Antennas and Propagation, 58(8), 2010.

[16]

L. J. Chu. Physical limitations of omni-directional antennas. Journal of Applied Physics, 1948.

[17]

G. Cohn, D. Morris, S. N. Patel, and D. S. Tan. Humantenna: Using the body as an antenna for real-time whole-body interaction. In ACM SIGCHI, 2012.

Digital Library

[18]

M. Cooke, P. Green, L. Josifovski, and A. Vizinho. Robust automatic speech recognition with missing and unreliable acoustic data. Speech Communication, 2001.

Digital Library

[19]

B. Dodd and R. Campbell. Hearing by eye: The psychology of lip-reading. Lawrence Erlbaum Associates, 1987.

[20]

P. Duchnowski, M. Hunke, D. Busching, U. Meier, and A. Waibel. Toward movement-invariant automatic lip-reading and speech recognition. In Proceedings of 1995 International Conference on Acoustics, Speech, and Signal Processing, 1995.

[21]

P. Duchnowski, U. Meier, and A. Waibel. See me, hear me: Integrating automatic speech recognition and lip-reading. In Proc. Int. Conf. Spoken Lang. Process, 1994.

[22]

I. A. Gheyas and L. S. Smith. Feature Subset Selection in Large Dimensionality Domains. Pattern Recognition, 2010.

Digital Library

[23]

D. Halperin, W. Hu, A. Shethy, and D. Wetherall. Predictable 802.11 packet delivery from wireless channel measurements. In ACM SIGCOMM, 2010.

Digital Library

[24]

C. Harrison, D. Tan, and D. Morris. Skinput: Appropriating the body as an input surface. In ACM SIGCHI, 2010.

Digital Library

[25]

Y. Jin, W. S. Soh, and W. C. Wong. Indoor localization with channel impulse response based fingerprint and nonparametric regression. IEEE Transactions on Wireless Communications, 9(3), 2010.

Digital Library

[26]

H. Junker, P. Lukowicz, and G. Troster. On the automatic segmentation of speech signals. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, 1987.

[27]

B. Kellogg, V. Talla, and S. Gollakota. Bringing gesture recognition to all devices. In USENIX NSDI, 2014.

Digital Library

[28]

K. Kumar, T. Chen, and R. M. Sternl. Profile view lip reading. In Proceedings of 2007 International Conference on Acoustics, Speech, and Signal Processing, 2007.

[29]

E. Larson, G. Cohn, S. Gupta, X. Ren, B. Harrison, D. Fox, and S. N. Patel. Heatwave: Thermal imaging for surface user interaction. In ACM SIGCHI, 2011.

Digital Library

[30]

K. C.-J. Lin, S. Gollakota, and D. Katabi. Random access heterogeneous mimo networks. In ACM SIGCOMM, 2011.

Digital Library

[31]

G. C. Martin. Preston blair phoneme series. http://www.garycmartin.com/mouth_shapes.html.

[32]

Q. Pu, S. Gupta, S. Gollakota, and S. Patel. Whole-home gesture recognition using wireless signals. In ACM MobiCom, 2013.

Digital Library

[33]

N. Saito and R. Coifman. Local discriminant bases and their applications. Journal of Mathematical Imaging and Vision, 5(4):337--358, 1995.

Digital Library

[34]

S. Salvador and P. Chan. Toward accurate dynamic time warping in linear time and space. Intell. Data Anal., 11(5), Oct. 2007.

Digital Library

[35]

M. Scholz, S. Sigg, H. R. Schmidtke, and M. Beigl. Challenges for device-free radio-based activity recognition. In Workshop on Context Systems, Design, Evaluation and Optimisation, 2011.

[36]

S. Sen, J. Lee, K.-H. Kim, and P. Congdon. Avoiding multipath to revive inbuilding wifi localization. In ACM MobiSys, 2013.

Digital Library

[37]

J. Wang and D. Katabi. Dude, where's my card? rfid positioning that works with multipath and non-line of sight. In ACM SIGCOMM, 2013.

Digital Library

[38]

J. R. Williams. Guidelines for the use of multimedia in instruction. In Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, 1998.

[39]

J. Wilson and N. Patwari. Radio Tomographic Imaging with Wireless Networks. IEEE Transactions on Mobile Computing, 9(5):621--632, 2010.

Digital Library

[40]

J. Xiao, K. Wu, Y. Yi, L. Wang, and L. Ni. Pilot: Passive Device-free Indoor Localization Using Channel State Information. In IEEE ICDCS, 2013.

Digital Library

[41]

Z. Yang, Z. Zhou, and Y. Liu. From RSSI to CSI: Indoor Localization via Channel Response. ACM Computing Surveys, 46(2):25:1--25:32, 2013.

Digital Library

[42]

M. Youssef, M. Mah, and A. Agrawala. Challenges: Device-free Passive Localization for Wireless Environments. In ACM MobiCom, 2007.

Digital Library

[43]

D. Zhang, J. Zhou, M. Guo, J. Cao, and T. Li. TASA: Tag-Free Activity Sensing Using RFID Tag Arrays. IEEE Transactions on Parallel and Distributed Systems, 22(4):558--570, 2011.

Digital Library

[44]

W. Zhang, X. Zhou, L. Yang, Z. Zhang, B. Y. Zhao, and H. Zheng. 3D Beamforming for Wireless Data Centers. In ACM Hotnets, 2011.

Digital Library

[45]

X. Zhou, Z. Zhang, Y. Zhu, Y. Li, S. Kumar, A. Vahdat, B. Y. Zhao, and H. Zheng. Mirror Mirror on the Ceiling: Flexible Wireless Links for Data Centers. In ACM SIGCOMM, 2012.

Digital Library

Cited By

Cai ZLi ZChen ZZhuo HZheng LWu XLiu Y(2024)Device-Free Wireless Sensing for Gesture Recognition Based on Complementary CSI Amplitude and PhaseSensors10.3390/s2411341424:11(3414)Online publication date: 25-May-2024
https://doi.org/10.3390/s24113414
Varga D(2024)Exposing Data Leakage in Wi-Fi CSI-Based Human Action Recognition: A Critical AnalysisInventions10.3390/inventions90400909:4(90)Online publication date: 15-Aug-2024
https://doi.org/10.3390/inventions9040090
Zhou ZLiu ZLiu YZhao YWang JZhang BXia YZhang XLi S(2024)TCS-Fall: Cross-individual fall detection system based on channel state information and time-continuous stack methodDIGITAL HEALTH10.1177/2055207624125904710Online publication date: 4-Jun-2024
https://doi.org/10.1177/20552076241259047
Show More Cited By

Index Terms

We can hear you with Wi-Fi!
1. Networks
  1. Network protocols

Recommendations

We Can Hear You with Wi-Fi!

Recent literature advances Wi-Fi signals to “see” people's motions and locations. This paper asks the following question: Can Wi-Fi “hear” our talks? We present WiHear, which enables Wi-Fi signals to “hear” our talks without deploying any devices. To ...
Wi-Fi Handbook: Building 802.11b Wireless Networks
Wi-fi backscatter: internet connectivity for RF-powered devices
SIGCOMM'14

RF-powered computers are small devices that compute and communicate using only the power that they harvest from RF signals. While existing technologies have harvested power from ambient RF sources (e.g., TV broadcasts), they require a dedicated gateway (...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MobiCom '14: Proceedings of the 20th annual international conference on Mobile computing and networking

September 2014

650 pages

ISBN:9781450327831

DOI:10.1145/2639108

General Chair:
Sung-Ju Lee
Narus Inc., USA
,
Program Chairs:
Ashutosh Sabharwal
Rice University, USA
,
Prasun Sinha
Ohio State University, USA

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 September 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Program for New Century Excellent Talents in University
Guangdong Natural Science Funds for Distinguished Young Scholar
Research Grants Council, University Grants Committee, Hong Kong
New Star of Pearl River on Science and Technology of Guangzhou
National Natural Science Foundation of China

Conference

MobiCom'14

Sponsor:

SIGMOBILE

MobiCom'14: The 20th Annual International Conference on Mobile Computing and Networking

September 7 - 11, 2014

Hawaii, Maui, USA

Acceptance Rates

MobiCom '14 Paper Acceptance Rate 36 of 220 submissions, 16%;

Overall Acceptance Rate 440 of 2,972 submissions, 15%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

206
Total Citations
View Citations
3,701
Total Downloads

Downloads (Last 12 months)142
Downloads (Last 6 weeks)12

Reflects downloads up to 24 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cai ZLi ZChen ZZhuo HZheng LWu XLiu Y(2024)Device-Free Wireless Sensing for Gesture Recognition Based on Complementary CSI Amplitude and PhaseSensors10.3390/s2411341424:11(3414)Online publication date: 25-May-2024
https://doi.org/10.3390/s24113414
Varga D(2024)Exposing Data Leakage in Wi-Fi CSI-Based Human Action Recognition: A Critical AnalysisInventions10.3390/inventions90400909:4(90)Online publication date: 15-Aug-2024
https://doi.org/10.3390/inventions9040090
Zhou ZLiu ZLiu YZhao YWang JZhang BXia YZhang XLi S(2024)TCS-Fall: Cross-individual fall detection system based on channel state information and time-continuous stack methodDIGITAL HEALTH10.1177/2055207624125904710Online publication date: 4-Jun-2024
https://doi.org/10.1177/20552076241259047
Liao QHuang YHuang YWu K(2024)An Eavesdropping System Based on Magnetic Side-Channel Signals Leaked by SpeakersACM Transactions on Sensor Networks10.1145/363706320:2(1-30)Online publication date: 10-Jan-2024
https://dl.acm.org/doi/10.1145/3637063
Yao YLi YZhu T(2024)Interference-Negligible Privacy-Preserved Shield for RF sensingIEEE Transactions on Mobile Computing10.1109/TMC.2023.3276930(1-12)Online publication date: 2024
https://doi.org/10.1109/TMC.2023.3276930
Luo JCao HJiang HYang YChen Z(2024)MIMOCrypt: Multi-User Privacy-Preserving Wi-Fi Sensing via MIMO Encryption2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00025(2812-2830)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00025
Chen SLin C(2024)Wi-Fi-Based Human Activity Recognition for Continuous, Whole-Room Monitoring of Motor Functions in Parkinson’s DiseaseIEEE Open Journal of Antennas and Propagation10.1109/OJAP.2024.33931175:3(788-799)Online publication date: Jun-2024
https://doi.org/10.1109/OJAP.2024.3393117
Hu PDuan SYang WGuo S(2024)WiMgrain: Multivariety Grain Moisture Content Detection Using Wi-Fi CSI DataIEEE Sensors Journal10.1109/JSEN.2024.337159624:8(12917-12933)Online publication date: 15-Apr-2024
https://doi.org/10.1109/JSEN.2024.3371596
Lv QDing YWang HJi Y(2024)Deep-Learning-Assisted Optical Transmitter Fingerprint Identification Based on the Constellation DiagramIEEE Internet of Things Journal10.1109/JIOT.2024.340620211:17(28635-28648)Online publication date: 1-Sep-2024
https://doi.org/10.1109/JIOT.2024.3406202
Luo FKhan SJiang BWu K(2024)Vision Transformers for Human Activity Recognition Using WiFi Channel State InformationIEEE Internet of Things Journal10.1109/JIOT.2024.337533711:17(28111-28122)Online publication date: 1-Sep-2024
https://doi.org/10.1109/JIOT.2024.3375337
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

ePub

View this article in ePub.

Media

Figures

Other

Tables

View Table of Contents