Just Accepted
ArmSpy++: Enhanced PIN Inference through Video-based Fine-grained Arm Posture Analysis
Authors: 
Huan Dai, Yuefeng Chen, Yicong Du, Luping Wang, Ziyu Shao, Hongbo Liu, Yanzhi Ren, Jiadi Yu, Bo LiuAuthors Info & Claims
Accepted on 03 September 2024
Abstract
As one of the most common ways for user authentication, Personal Identification Number (PIN), due to its simplicity and convenience, has suffered from plenty of side-channel attacks, which pose a severe threat to people’s privacy and property. The success of existing attacks is usually built upon the premise of no occlusion between the attacker and the victim’s hand gesture, but it increases the difficulty of launching the attack and the possibility of exposure. To overcome such limitation, we propose ArmSpy++, an improved video-assisted PIN inference attack built upon our previous research, ArmSpy. Specifically, ArmSpy++ employs new modules to leverage more features like the keystroke-induced elbow bending, wrist speed variation, and the spatial relationship between different arm joints, to correctly detect Keystrokes. ArmSpy++ delves into the perspective relationship and natural typing habits to ensure a high success rate of PIN inference. We also re-designed the inferred PIN pattern coordination mechanism to accurately deduce the PINs. By using a pre-trained HigherHRNet model for posture estimation ArmSpy++ eliminates the necessity of additional training. The extensive experiments demonstrate that ArmSpy++ can achieve over \(83.1\%\) average accuracy with 3 attempts and even \(92.5\%\) for some victims, indicating the severity of the threat posed by ArmSpy++.
References
[1]
Kamran Ali, Alex X Liu, Wei Wang, and Muhammad Shahzad. 2015. Keystroke recognition using WiFi signals. In Proceedings of Annual International Conference on Mobile Computing and Networking. 90–102.
[2]
Dmitri Asonov and Rakesh Agrawal. 2004. Keyboard acoustic emanations. In IEEE Symposium on Security and Privacy. IEEE, 3–11.
[3]
Liang Cai and Hao Chen. 2011. TouchLogger: Inferring keystrokes on touch screen from smartphone motion. USENIX Workshop on Hot Topics in Security 11, 2011 (2011), 9.
[4]
Matteo Cardaioli, Stefano Cecconello, Mauro Conti, Simone Milani, Stjepan Picek, and Eugen Saraci. 2022. Hand Me Your PIN! Inferring ATM PINs of Users Typing with a Covered Hand. In 31st USENIX Security Symposium (USENIX Security 22). 1687–1704.
[5]
Yuefeng Chen, Yicong Du, Chunlong Xu, Yanghai Yu, Hongbo Liu, Huan Dai, Yanzhi Ren, and Jiadi Yu. 2022. ArmSpy: Video-assisted PIN Inference Leveraging Keystroke-induced Arm Posture Changes. In IEEE INFOCOM 2022-IEEE Conference on Computer Communications. IEEE, 1878–1887.
[6]
Yimin Chen, Tao Li, Rui Zhang, Yanchao Zhang, and Terri Hedgpeth. 2018. EyeTell: Video-assisted touchscreen keystroke inference from eye movements. In IEEE Symposium on Security and Privacy. IEEE, 144–160.
[7]
Bowen Cheng, Bin Xiao, Jingdong Wang, Honghui Shi, Thomas S. Huang, and Lei Zhang. 2020. HigherHRNet: Scale-aware representation learning for bottom-Up human pose estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
[8]
Gerson de Souza Faria and Hae Yong Kim. 2019. Differential audio analysis: A new side-channel attack on PIN pads. International Journal of Information Security 18, 1 (2019), 73–84.
[9]
Chenning Li, Manni Liu, and Zhichao Cao. 2020. WiHF: Enable user identified gesture recognition with WiFi. In IEEE Conference on Computer Communications. 586–595. https://doi.org/10.1109/INFOCOM41043.2020.9155539
[10]
Mengyuan Li, Yan Meng, Junyi Liu, Haojin Zhu, Xiaohui Liang, Yao Liu, and Na Ruan. 2016. When CSI meets public WiFi: Inferring your mobile phone password via WiFi signals. In Proceedings of ACM SIGSAC Conference on Computer and Communications Security. 1068–1079.
[11]
Zhengxiong Li, Fenglong Ma, Aditya Singh Rathore, Zhuolin Yang, Baicheng Chen, Lu Su, and Wenyao Xu. 2020. Wavespy: Remote and through-wall screen attack via mmwave sensing. In 2020 IEEE Symposium on Security and Privacy (SP). IEEE, 217–232.
[12]
Ximing Liu, Yingjiu Li, and Robert H Deng. 2021. UltraPIN: Inferring PIN Entries via Ultrasound. In Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security. 944–957.
[13]
Xiangyu Liu, Zhe Zhou, Wenrui Diao, Zhou Li, and Kehuan Zhang. 2015. When good becomes evil: Keystroke inference with smartwatch. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. 1273–1285.
[14]
Anindya Maiti, Murtuza Jadliwala, Jibo He, and Igor Bilogrevic. 2015. (Smart) watch your taps: Side-channel keystroke inference attacks using smartwatches. In Proceedings of ACM International Symposium on Wearable Computers. 27–30.
[15]
Alfred J Menezes, Paul C Van Oorschot, and Scott A Vanstone. 2018. Handbook of applied cryptography. CRC press.
[16]
Emmanuel Owusu, Jun Han, Sauvik Das, Adrian Perrig, and Joy Zhang. 2012. Accessory: Password inference using accelerometers on smartphones. In Proceedings of Workshop on Mobile Computing Systems & Applications. 1–6.
[17]
Sai Deepika Regani, Beibei Wang, Min Wu, and K. J. Ray Liu. 2020. Time reversal based robust gesture recognition using WiFi. In IEEE International Conference on Acoustics, Speech and Signal Processing. 8309–8313. https://doi.org/10.1109/ICASSP40776.2020.9053420
[18]
Mohd Sabra, Anindya Maiti, and Murtuza Jadliwala. 2018. Keystroke inference using ambient light sensor on wrist-wearables: a feasibility study. In Proceedings of the 4th ACM Workshop on Wearable Systems and Applications. 21–26.
[19]
Mohd Sabra, Anindya Maiti, and Murtuza Jadliwala. 2020. Zoom on the keystrokes: exploiting video calls for keystroke inference attacks. arXiv preprint arXiv:2010.12078(2020).
[20]
Abraham Savitzky and Marcel JE Golay. 1964. Smoothing and differentiation of data by simplified least squares procedures. Analytical Chemistry 36, 8 (1964), 1627–1639.
[21]
Sheng Shen, He Wang, and Romit Roy Choudhury. 2016. I am a smartwatch and i can track my user’s arm. In Proceedings of the 14th annual international conference on Mobile systems, applications, and services. 85–96.
[22]
Diksha Shukla, Rajesh Kumar, Abdul Serwadda, and Vir V Phoha. 2014. Beware, your hands reveal your secrets!. In Proceedings of ACM SIGSAC Conference on Computer and Communications Security. 904–917.
[23]
Jingchao Sun, Xiaocong Jin, Yimin Chen, Jinxue Zhang, Yanchao Zhang, and Rui Zhang. 2016. Visible: Video-assisted keystroke inference from tablet backside motion. In Network and Distributed System Security Symposium.
[24]
Carlo Tomasi and Roberto Manduchi. 1998. Bilateral filtering for gray and color images. In International Conference on Computer Vision. IEEE, 839–846.
[25]
Jaikumar Vijayan. 2007. TJX data breach: At 45.6M card numbers, it’s the biggest ever. https://www.computerworld.com/article/2544306/tjx-data-breach--at-45-6m-card-numbers--it-s-the-biggest-ever.html.
[26]
Martin Vuagnoux and Sylvain Pasini. 2009. Compromising electromagnetic emanations of wired and wireless keyboards. In USENIX Security Symposium, Vol. 8. 1–16.
[27]
Chen Wang, Xiaonan Guo, Yan Wang, Yingying Chen, and Bo Liu. 2016. Friend or foe? Your wearable devices reveal your personal pin. In Proceedings of the 11th ACM on Asia Conference on Computer and Communications Security. 189–200.
[28]
Zhi Xu, Kun Bai, and Sencun Zhu. 2012. Taplogger: Inferring user inputs on smartphone touchscreens using on-board motion sensors. In Proceedings of ACM Conference on Security and Privacy in Wireless and Mobile Networks. 113–124.
[29]
Qinggang Yue, Zhen Ling, Xinwen Fu, Benyuan Liu, Kui Ren, and Wei Zhao. 2014. Blind recognition of touched keys on mobile devices. In Proceedings of ACM SIGSAC Conference on Computer and Communications Security. 1403–1414.
[30]
Tianming Zhao, Jian Liu, Yan Wang, Hongbo Liu, and Yingying Chen. 2018. PPG-based finger-level gesture recognition leveraging wearables. In IEEE INFOCOM 2018-IEEE Conference on Computer Communications. IEEE, 1457–1465.
Index Terms
- ArmSpy++: Enhanced PIN Inference through Video-based Fine-grained Arm Posture Analysis
Index terms have been assigned to the content through auto-classification.
Recommendations
ArmSpy: Video-assisted PIN Inference Leveraging Keystroke-induced Arm Posture Changes
IEEE INFOCOM 2022 - IEEE Conference on Computer CommunicationsPIN inference attack leveraging keystroke-induced side-channel information poses a substantial threat to the security of people’s privacy and properties. Among various PIN inference attacks, video-assisted method provide more intuitive and robust ...
Problems Encountered in Seated Arm Reach Posture Reconstruction: Need for a More Realistic Spine and Upper Limb Kinematic Model
ICDHM '09: Proceedings of the 2nd International Conference on Digital Human Modeling: Held as Part of HCI International 2009In this paper, we will present the main problems encountered for reconstructing in-vehicle reach postures. Among the 2176 successfully captured movements, about 7.4% of them were considered as "bad quality" with a high residual error between ...
Gender-based analysis of muscle activation while in a sedentary work posture
BACKGROUND:Many people work in a sitting position where they have to use their upper extremities and hands. Muscle activity in sitting position is affected by the chair height of, the height of the worktable, and the distance to the working target.
...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
EISSN:2471-2574
Table of ContentsCopyright © 2024 Copyright held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Online AM: 23 September 2024
Accepted: 03 September 2024
Revised: 18 July 2024
Received: 06 September 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 23Total Downloads
- Downloads (Last 12 months)23
- Downloads (Last 6 weeks)23
Reflects downloads up to 26 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in