RFSpy: Eavesdropping on Online Conversations with Out-of-Vocabulary Words by Sensing Metal Coil Vibration of Headsets Leveraging RFID
Authors: 
Yunzhong Chen, Jiadi Yu, Yingying Chen, Linghe Kong, 
Yanmin Zhu, Yi-Chao ChenAuthors Info & Claims
Pages 169 - 182
Abstract
Eavesdropping on human sound is one of the most common but harmful ways to threaten personal privacy. As one of the most essential accessories, headsets have been widely used in common online conversations, such as online calls, video meetings, etc. The metal coil vibration patterns of headset speakers/microphones have been proven to be highly correlated with the speaker-produced/microphone-received sound content. This paper presents an online conversation eavesdropping system, RFSpy, which uses only one RFID tag attached on a headset to alternately sense the metal coil vibrations of headset speaker and microphone for eavesdropping on speaker-produced and microphone-received sound. In some accessible scenarios, such as meeting rooms, offices, etc., assuming attackers secretly attach a small, battery-free RFID tag under one ear cushion of an eavesdropped user's headset without being noticed. Meanwhile, RFID readers are camouflaged as decorations placed in/out of rooms to transmit and receive RF signals. When the eavesdropped user talks with other users online by using the headset, RFSpy first activates the RFID tag attached on the headset to capture the metal coil vibration patterns of headset speaker and microphone upon RF signals. Then, RFSpy reconstructs sound spectrograms from the RF signal-based vibration patterns for not only trained words but also untrained (i.e., out-of-vocabulary) words by utilizing a designed Sound Spectrogram Reconstruction (SSR) network. Finally, RFSpy converts the sound spectrograms to conversation content through a sound recognition API. Extensive experiments in real environments demonstrate that RFSpy can eavesdrop on online conversations with out-of-vocabulary (OOV) words effectively.
References
[1]
Zhongjie Ba, Tianhang Zheng, Xinyu Zhang, Zhan Qin, Baochun Li, Xue Liu, and Kui Ren. 2020. Learning-based Practical Smartphone Eavesdropping with Built-in Accelerometer. In Proceedings of NDSS'20. San Diego, USA.
[2]
Yang Bai, Li Lu, Jerry Cheng, Jian Liu, Yingying Chen, and Jiadi Yu. 2020. Acoustic-based sensing and applications: A survey. Computer Networks 181 (2020), 107447.
[3]
Suryoday Basak and Mahanth Gowda. 2022. mmspy: Spying phone calls using mmwave radars. In Proceedings of IEEE S&P'22. San Francisco, USA.
[4]
Steven Boll. 1979. Suppression of acoustic noise in speech using spectral subtraction. IEEE Transactions on acoustics, speech, and signal processing 27, 2 (1979), 113--120.
[5]
BUGHUNTER. 2023. Microphone jammer BugHunter DAudio bda-3 Voices with 7 ultrasonic transducers. [Online]. Available: https://www.amazon.com/BDA-3-Control-Microphone-Suppressor-Recording/dp/B089WH2B9F.
[6]
Yetong Cao, Fan Li, Huijie Chen, Xiaochen Liu, Chunhui Duan, and Yu Wang. 2023. I Can Hear You Without a Microphone: Live Speech Eavesdropping From Earphone Motion Sensors. In Proceedings of IEEE INFOCOM'23. New York, USA.
[7]
Yunzhong Chen, Jiadi Yu, Linghe Kong, Hao Kong, Yanmin Zhu, and Yi-Chao Chen. 2023. RF-Mic: Live Voice Eavesdropping via Capturing Subtle Facial Speech Dynamics Leveraging RFID. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 2 (2023), 1--25.
[8]
Yunzhong Chen, Jiadi Yu, Linghe Kong, Yanmin Zhu, and Feilong Tang. 2022. RFPass: Towards environment-independent gait-based user authentication leveraging RFID. In Proceedings of IEEE SECON'22. Virtual Conference.
[9]
Google cloud. 2023. Cloud speech-to-text documentation. [Online]. Available: https://cloud.google.com/speech-to-text.
[10]
Antonie Cohen. 1965. The phonemes of English. Springer.
[11]
Israel Cohen, Yiteng Huang, Jingdong Chen, and Jacob Benesty. 2009. Pearson correlation coefficient. Noise reduction in speech processing (2009), 1--4.
[12]
M Dobhn Daniel et al. 2008. The rf in rfid passive uhf rfid in practice. In Elsevier.
[13]
R Dauman. 2013. Bone conduction: an explanation for this phenomenon comprising complex mechanisms. European annals of otorhinolaryngology, head and neck diseases 130, 4 (2013), 209--213.
[14]
Kevin D'hoe, Van Nieuwenhuyse, et al. 2009. Influence of different types of metal plates on a high frequency RFID loop antenna: study and design. Advances in electrical and computer engineering 9, 2 (2009), 3--8.
[15]
Konstantin Dragomiretskiy and Dominique Zosso. 2013. Variational mode decomposition. IEEE transactions on signal processing 62, 3 (2013), 531--544.
[16]
Long Fan, Lei Xie, Xinran Lu, Yi Li, Chuyu Wang, and Sanglu Lu. 2023. mmMIC: Multi-modal Speech Recognition based on mmWave Radar. In Proceedings of IEEE INFOCOM'23. New York, USA.
[17]
Chao Feng, Jie Xiong, Liqiong Chang, Fuwei Wang, Ju Wang, and Dingyi Fang. 2021. Rf-identity: Non-intrusive person identification based on commodity rfid devices. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 5. 1--23.
[18]
Thomas Gay. 1981. Mechanisms in the control of speech rate. Phonetica 38, 1-3 (1981), 148--158.
[19]
Dedre Gentner and Arthur B Markman. 2006. Defining structural similarity. The Journal of Cognitive Science 6, 1 (2006), 1--20.
[20]
Daniel Griffin and Jae Lim. 1984. Signal estimation from modified short-time Fourier transform. IEEE Transactions on acoustics, speech, and signal processing 32, 2 (1984), 236--243.
[21]
Yejun He and Zhengzheng Pan. 2013. Design of UHF RFID broadband antimetal tag antenna applied on surface of metallic objects. In Proceedings of IEEE WCNC'13. Glasgow, UK.
[22]
Pengfei Hu, Wenhao Li, Riccardo Spolaor, and Xiuzhen Cheng. 2023. mmecho: A mmwave-based acoustic eavesdropping method. In Proceedings of IEEE S&P'22. Delft, Netherlands.
[23]
Pengfei Hu, Yifan Ma, Panneer Selvam Santhalingam, Parth H Pathak, and Xiuzhen Cheng. 2022. Milliear: Millimeter-wave acoustic eavesdropping with unconstrained vocabulary. In Proceedings of IEEE INFOCOM'22. Virtual Conference.
[24]
Pengfei Hu, Hui Zhuang, Panneer Selvam Santhalingam, Riccardo Spolaor, Parth Pathak, Guoming Zhang, and Xiuzhen Cheng. 2022. Accelerometer acoustic eavesdropping with unconstrained vocabulary. In Proceedings of IEEE S&P. San Francisco, USA.
[25]
F. Mavromatis N. Kargas and A. Bletsas. 2019. USRP reader. [Online]. Available: https://github.com/nkargas/Gen2-UHF-RFID-Reader.
[26]
Jung Ho Kim, Jim Tai Kim, Jin O Kim, and Jin Ki Min. 1997. Acoustic characteristics of a loudspeaker obtained by vibration and acoustic analyses. WIT Transactions on The Built Environment 28 (1997), 181--190.
[27]
Hao Kong, Li Lu, Jiadi Yu, Yingying Chen, and Feilong Tang. 2020. Continuous authentication through finger gesture interaction for smart homes using WiFi. IEEE Transactions on Mobile Computing 20, 11 (2020), 3148--3162.
[28]
Hao Kong, Xiangyu Xu, Jiadi Yu, Qilin Chen, Chenguang Ma, Yingying Chen, Yi-Chao Chen, and Linghe Kong. 2022. m3track: mmWave-based Multi-user 3D Posture Tracking. In Proc. of ACM MobiSys. Portland, OR, USA.
[29]
Ping Li, Zhenlin An, Lei Yang, and Panlong Yang. 2019. Towards physical-layer vibration sensing with rfids. In Proceedings of IEEE INFOCOM'19. Paris, France.
[30]
Ping Li, Zhenlin An, Lei Yang, Panlong Yang, and QiongZheng Lin. 2019. RFID harmonic for vibration sensing. IEEE Transactions on Mobile Computing 20, 4 (2019), 1614--1626.
[31]
Qianru Liao, Yongzhi Huang, Yandao Huang, Yuheng Zhong, Huitong Jin, and Kaishun Wu. 2022. MagEar: eavesdropping via audio recovery using magnetic side channel. In Proceedings of ACM MobiSys'22. Portland, Oregon.
[32]
Weifeng Liu, Puskal P Pokharel, and Jose C Principe. 2008. The kernel least-mean-square algorithm. IEEE Transactions on signal processing 56, 2 (2008), 543--554.
[33]
Héctor A Cordourier Maruri, Paulo Lopez-Meyer, Jonathan Huang, Willem Marco Beltman, Lama Nachman, and Hong Lu. 2018. V-Speech: noise-robust speech capturing glasses using vibration sensors. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 2. 1--23.
[34]
Yan Michalevsky, Dan Boneh, and Gabi Nakibly. 2014. Gyrophone: Recognizing speech from gyroscope signals. In Proceedings of USENIX'14. San Diego, USA.
[35]
Philip McCord Morse, Acoustical Society of America, and American Institute of Physics. 1948. Vibration and sound. McGraw-Hill New York.
[36]
rfidhy. 2022. The Smallest RFID Tag as Thin as Sand. [Online]. Available: https://www.rfidhy.com/the-smallest-rfid-tag-as-thin-as-sand/.
[37]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of IEEE CVPR'22. New Orleans, USA.
[38]
Sriram Sami, Yimin Dai, Sean Rui Xiang Tan, Nirupam Roy, and Jun Han. 2020. Spying with your robot vacuum cleaner: eavesdropping via lidar sensors. In Proceedings of ACM SenSys'20. Yokohama, Japan.
[39]
Cong Shi, Xiangyu Xu, Tianfang Zhang, Payton Walker, Yi Wu, Jian Liu, Nitesh Saxena, Yingying Chen, and Jiadi Yu. 2021. Face-Mic: inferring live speech and speaker identity via subtle facial dynamics captured by AR/VR motion sensors. In Proceedings of ACM MobiCom '21. New Orleans, USA.
[40]
Weigao Su, Daibo Liu, Taiyuan Zhang, and Hongbo Jiang. 2021. Towards device independent eavesdropping on telephone conversations with built-in accelerometer. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 5. 1--29.
[41]
Richard EA van Emmerik and Erwin EH van Wegen. 2000. On variability and stability in human movement. Journal of Applied Biomechanics 16, 4 (2000), 394--406.
[42]
Gianfranco Andía Vera, Yvan Duroc, and Smail Tedjini. 2013. Analysis of harmonics in UHF RFID signals. IEEE Transactions on Microwave Theory and Techniques 61, 6 (2013), 2481--2490.
[43]
Chao Wang, Feng Lin, Zhongjie Ba, Fan Zhang, Wenyao Xu, and Kui Ren. 2022. Wavesdropper: Through-wall Word Detection of Human Speech via Commercial mmWave Devices. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 6. 1--26.
[44]
Chao Wang, Feng Lin, Tiantian Liu, Ziwei Liu, Yijie Shen, Zhongjie Ba, Li Lu, Wenyao Xu, and Kui Ren. 2022. mmphone: Acoustic eavesdropping on loudspeakers via mmwave-characterized piezoelectric effect. In Proceedings of IEEE INFOCOM'22. Virtual Conference.
[45]
Chao Wang, Feng Lin, Tiantian Liu, Kaidi Zheng, Zhibo Wang, Zhengxiong Li, Ming-Chun Huang, Wenyao Xu, and Kui Ren. 2022. mmEve: eavesdropping on smartphone's earpiece via COTS mmWave device. In Proceedings of ACM MobiCom'22. Sydney, Australia.
[46]
Chuyu Wang, Lei Xie, Yuancan Lin, Wei Wang, Yingying Chen, Yanling Bu, Kai Zhang, and Sanglu Lu. 2021. Thru-the-wall eavesdropping on loudspeakers via RFID by capturing sub-mm level vibration. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 5. 1--25.
[47]
Chuyu Wang, Lei Xie, Wei Wang, Yingying Chen, Yanling Bu, and Sanglu Lu. 2018. Rf-ecg: Heart rate variability assessment based on cots rfid tag array. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 2. 1--26.
[48]
Teng Wei, Shu Wang, Anfu Zhou, and Xinyu Zhang. 2015. Acoustic eavesdropping through wireless vibrometry. In Proceedings of ACM MobiCom'15. Paris, France.
[49]
Fu Xiao, Zhongqin Wang, Ning Ye, Ruchuan Wang, and Xiang-Yang Li. 2017. One more tag enables fine-grained RFID localization and tracking. IEEE/ACM Transactions on Networking 26, 1 (2017), 161--174.
[50]
Binbin Xie, Jie Xiong, Xiaojiang Chen, and Dingyi Fang. 2020. Exploring commodity rfid for contactless sub-millimeter vibration sensing. In Proceedings of ACM SenSys'20. Yokohama, Japan.
[51]
Chenhan Xu, Zhengxiong Li, Hanbin Zhang, Aditya Singh Rathore, Huining Li, Chen Song, Kun Wang, and Wenyao Xu. 2019. Waveear: Exploring a mmwave-based noise-resistant speech sensing for voice-user interface. In Proceedings of ACM MobiSys'19. Seoul, South Korea.
[52]
Xiangyu Xu, Jiadi Yu, Yingying Chen, Qin Hua, Yanmin Zhu, Yi-Chao Chen, and Minglu Li. 2020. TouchPass: Towards Behavior-irrelevant On-touch User Authentication on Smartphones Leveraging Vibrations. In Proc. of ACM MobiCom. London, UK.
[53]
Xiangyu Xu, Jiadi Yu, Yingying Chen, Yanmin Zhu, Linghe Kong, and Minglu Li. 2019. BreathListener: Fine-grained Breathing Monitoring in Driving Environments Utilizing Acoustic Signals. In Proc. of ACM MobiSys. Seoul, South Korea.
[54]
Chen Yan, Guoming Zhang, Xiaoyu Ji, Tianchen Zhang, Taimin Zhang, and Wenyuan Xu. 2019. The feasibility of injecting inaudible voice commands to voice assistants. IEEE Transactions on Dependable and Secure Computing 18, 3 (2019), 1108--1124.
[55]
Lei Yang, Yao Li, Qiongzheng Lin, Huanyu Jia, Xiang-Yang Li, and Yunhao Liu. 2017. Tagbeat: Sensing mechanical vibration period with cots rfid systems. IEEE/ACM transactions on networking 25, 6 (2017), 3823--3835.
[56]
Lei Yang, Yao Li, Qiongzheng Lin, Xiang-Yang Li, and Yunhao Liu. 2016. Making sense of mechanical vibration period with sub-millisecond accuracy using backscatter signals. In Proceedings of ACM MobiCom'16. New York, USA.
[57]
Lei Yang, Qiongzheng Lin, Xiangyang Li, Tianci Liu, and Yunhao Liu. 2015. See through walls with COTS RFID system. In Proceedings of ACM Mobicom'15. Paris, France.
[58]
Panlong Yang, Yuanhao Feng, Jie Xiong, Ziyang Chen, and Xiang-Yang Li. 2020. Rf-ear: Contactless multi-device vibration sensing and identification using cots rfid. In Proceedings of IEEE INFOCOM'20. Virtual Conference.
[59]
Chuyu Wang Lei Xie Jingyi Ning Yiwen Feng, Kai Zhang and Shijia Chen. 2023. mmEavesdropper: Signal Augmentation-based Directional Eavesdropping with mmWave Radar. In Proceedings of IEEE INFOCOM'23. New York, USA.
[60]
Jiadi Yu, Li Lu, Yingying Chen, Yanmin Zhu, and Linghe Kong. 2019. An indirect eavesdropping attack of keystrokes on touch screen through acoustic sensing. IEEE Transactions on Mobile Computing 20, 2 (2019), 337--351.
[61]
Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J. Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu. 2019. LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech. In Proceedings of IEEE ISCA'19. ISCA, Graz, Austria.
[62]
Li Zhang, Parth H Pathak, Muchen Wu, Yixin Zhao, and Prasant Mohapatra. 2015. Accelword: Energy efficient hotword detection through accelerometer. In Proceedings of ACM MobiSys'15. Florence, Italy.
[63]
Shijia Zhang, Yilin Liu, and Mahanth Gowda. 2023. I Spy You: Eavesdropping Continuous Speech on Smartphones via Motion Sensors. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 6. 1--31.
[64]
Yongzhao Zhang, Wei-Hsiang Huang, Chih-Yun Yang, Wen-Ping Wang, Yi-Chao Chen, Chuang-Wen You, Da-Yuan Huang, Guangtao Xue, and Jiadi Yu. 2020. Endophasia: Utilizing acoustic-based imaging for issuing contact-free silent speech commands. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1--26.
[65]
Yanmin Zhu, Ruobing Jiang, Jiadi Yu, Zhi Li, and Minglu Li. 2014. Geographic routing based on predictive locations in vehicular ad hoc networks. EURASIP Journal on Wireless Communications and Networking 2014 (2014), 1--9.
Index Terms
- RFSpy: Eavesdropping on Online Conversations with Out-of-Vocabulary Words by Sensing Metal Coil Vibration of Headsets Leveraging RFID
Recommendations
Thru-the-wall Eavesdropping on Loudspeakers via RFID by Capturing Sub-mm Level Vibration
The unprecedented success of speech recognition methods has stimulated the wide usage of intelligent audio systems, which provides new attack opportunities for stealing the user privacy through eavesdropping on the loudspeakers. Effective eavesdropping ...
On the security of RFID anti-counting security protocol (ACSP)
Recently Qian et al. (2012) [26] have proposed a new attack for RFID systems, called counting attack, where the attacker just aims to estimate the number of tagged objects instead of steal the tags' private information. They have stated that most of ...
On two RFID privacy notions and their relations
Privacy of RFID systems is receiving increasing attention in the RFID community. Basically, there are two kinds of RFID privacy notions in the literature: one based on the indistinguishability of two tags, denoted as ind-privacy, and the other based on ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
June 2024
778 pages
ISBN:9798400705816
DOI:10.1145/3643832
- Chairs:
- Tadashi Okoshi,
- JeongGil Ko,
- Program Chair:
- Robert LiKamWa
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
MOBISYS '24
Sponsor:
MOBISYS '24: 22nd Annual International Conference on Mobile Systems, Applications and Services
June 3 - 7, 2024
Minato-ku, Tokyo, Japan
Acceptance Rates
Overall Acceptance Rate 274 of 1,679 submissions, 16%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 209Total Downloads
- Downloads (Last 12 months)209
- Downloads (Last 6 weeks)59
Reflects downloads up to 26 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in