Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3055031.3055088acmotherconferencesArticle/Chapter ViewAbstractPublication PagescpsweekConference Proceedingsconference-collections
research-article

Pitchln: eavesdropping via intelligible speech reconstruction using non-acoustic sensor fusion

Published: 18 April 2017 Publication History

Abstract

Despite the advent of numerous Internet-of-Things (IoT) applications, recent research demonstrates potential side-channel vulnerabilities exploiting sensors which are used for event and environment monitoring. In this paper, we propose a new side-channel attack, where a network of distributed non-acoustic sensors can be exploited by an attacker to launch an eavesdropping attack by reconstructing intelligible speech signals. Specifically, we present PitchIn to demonstrate the feasibility of speech reconstruction from non-acoustic sensor data collected offline across networked devices. Unlike speech reconstruction which requires a high sampling frequency (e.g., > 5 KHz), typical applications using non-acoustic sensors do not rely on richly sampled data, presenting a challenge to the speech reconstruction attack. Hence, PitchIn leverages a distributed form of Time Interleaved Analog-Digital-Conversion (TIADC) to approximate a high sampling frequency, while maintaining low per-node sampling frequency. We demonstrate how distributed TI-ADC can be used to achieve intelligibility by processing an interleaved signal composed of different sensors across networked devices. We implement PitchIn and evaluate reconstructed speech signal intelligibility via user studies. PitchIn has word recognition accuracy as high as 79%. Though some additional work is required to improve accuracy, our results suggest that eavesdropping using a fusion of non-acoustic sensors is a real and practical threat.

References

[1]
ADXL103/ADXL203 Datasheet. Analog Devices Data Sheet.
[2]
Arduino Ethernet Shield. https://www.arduino.cc/en/Main/ArduinoEthernetShield.
[3]
Arduino FAT16/FAT32 Library. https://github.com/greiman/SdFat.
[4]
Arduino/Genuino UNO. https://www.arduino.cc/en/Main/arduinoBoardUno.
[5]
LIS331DLH: MEMS digital output motion sensor ultra low-power high performance 3-axes "nano" accelerometer. STMicroelectronics Datasheet.
[6]
PTPd. http://ptpd.sourceforge.net/.
[7]
Spencer Ackerman and James Ball. 2014. Optic Nerve: millions of Yahoo webcam images intercepted by GCHQ. http://www.theguardian.com/world/2014/feb/27/gchq-nsa-webcam-images-internet-yahoo.
[8]
Amazon. 2015. Amazon Mechanical Truk. https://www.mturk.com/mturk/welcome.
[9]
Tali Arbel. 2016. Verizon Business Accounts Hacked. http://www.theguardian.com/world/2014/feb/27/gchq-nsa-webcam-images-internet-yahoo.
[10]
Atmel. Atmel 8-bit AVR Microcontroller with 4/8/16/32K Bytes In-System Programmable Flash. Atmel. http://www.atmel.com/Images/doc8161.pdf.
[11]
Audacity. DC Offset. http://manual.audacityteam.org/o/man/dc_offset.html.
[12]
S. Butterworth. 1930. On the Theory of Filter Amplifiers. Experimental Wireless and the Wireless Engineer 7 (1930), 536--541.
[13]
IO Sensor Nederland b.v. SM-24 Geophone Element. http://cdn.sparkfun.com/datasheets/Sensors/Accelerometers/SM-24%20Brochure.pdf.
[14]
Abe Davis, Michael Rubinstein, Neal Wadhwa, Gautham Mysore, Fredo Durand, and William T. Freeman. 2014. The Visual Microphone: Passive Recovery of Sound from Video. ACM Transactions on Graphics (Proc. SIGGRAPH) 33, 4 (2014), 79:1--79:10.
[15]
Analog Devices. ADXL335 Datasheet. http://www.analog.com/media/en/technical-documentation/data-sheets/ADXL335.pdf.
[16]
Analog Devices. LG Smart TV platform guide. http://docs.madewithmarmalade.com/display/MD/LG+Smart+TV+platform+guide.
[17]
M. El-Chammas and B.Murmann. 2011. A 12-GS/s81-mW 5-bit Time-Interleaved Flash ADC With Background Timing Skew Calibration. IEEE Journal of Solid-State Circuits 46, 4 (April 2011), 838--847.
[18]
Chuck Goudie and Ross Weidner. 2015. Home hackers: Digital invaders a threat to your house. http://abc7chicago.com/technology/home-hackers-digital-invaders-a-threat-to-your-house/515520/.
[19]
Jun Han, Emmanuel Owusu, Le T. Nguyen, Adrian Perrig, and Joy Zhang. 2012. ACComplice: Location inference using accelerometers on smartphones. In Communication Systems and Networks (COMSNETS).
[20]
T. Hao, R. Zhou, G. Xing, and M. Mutka. 2011. WizSync: Exploiting Wi-Fi Infrastructure for Clock Synchronization in Wireless Sensor Networks. In Real-Time Systems Symposium (RTSS), 2011 IEEE 32nd.
[21]
Chris Heinonen. 2015. Your Privacy, Your Devices, and You. http://thewirecutter.com/blog/your-privacy-your-devices-and-you/.
[22]
Texas Instruments. 2011. ADC081000, ADC08D1000: Interleaving ADCs for Higher Sample Rates. Texas Instruments Whitepaper SNAA111.
[23]
Texas Instruments. 2014. LMV3xx Low-Voltage Rail-to-Rail Output Operational Amplifiers.
[24]
Jungsuk Kim and Ian Lane. 2014. Accelerating large vocabulary continuous speech recognition on heterogeneous CPU-GPU platforms. In Acoustics, Speech and Signal Processing (ICASSP), IEEE. 6854209
[25]
Sukun Kim, S. Pakzad, D. Culler, J. Demmel, G. Fenves, S. Glaser, and M. Turon. 2007. Health Monitoring of Civil Infrastructures Using Wireless Sensor Networks. In Information Processing in Sensor Networks (IPSN).
[26]
Wise Lab. FireFly3. "http://wise.ece.cmu.edu/redmine/projects/firefly/wiki/FireFly3_x2".
[27]
Sapna Maheshwari. 2017. Is Your Vizio Television Spying on You? What to Know. https://nyti.ms/2kKRKpS.
[28]
Rahul Mangharam, Anthony Rowe, and Raj Rajkumar. 2007. FireFly: A Crosslayer Platform for Real-time Embedded Wireless Networks. Real-Time Syst. 37, 3 (Dec. 2007), 183--231.
[29]
Chung-Tse Mar, Mat Hans, Mark Smith, Tajana Simunic, and Ronald Schafer. 2002. A High-Quality, Energy Optimized, Real-Time Sampling Rate Conversion Library for the StrongARM Microprocessor. Technical Report HPL-2002--159.
[30]
Philip Marquardt, Arunabh Verma, Henry Carter, and Patrick Traynor. 2011. (Sp)iPhone: Decoding Vibrations from Nearby Keyboards Using Mobile Phone Accelerometers. In Conference on Computer and Communications Security (CCS). ACM, New York, NY, USA.
[31]
Chris Matyszczyk. 2015. Samsung's warning: Our Smart TVs record your living room chatter. https://www.cnet.com/news/samsungs-warning-our-smart-tvs-record-your-living-room-chatter/.
[32]
Yan Michalevsky, Dan Boneh, and Gabi Nakibly. 2014. Gyrophone: Recognizing Speech from Gyroscope Signals. In 23rd USENIX Security Symposium (USENIX Security 14). USENIX Association, San Diego, CA, 1053--1067. https://www.usenix.org/conference/usenixsecurity14/technical-sessions/presentation/michalevsky
[33]
Steven Norton. 2015. Internet of Things Market to Reach $1.7 Trillion by 2020. http://blogs.wsj.com/cio/2015/06/02/internet-of-things-market-to-reach-1-7-trillion-by-2020-idc/.
[34]
Emmanuel Owusu, Jun Han, Sauvik Das, Adrian Perrig, and Joy Zhang. 2012. AC-Cessory: Password Inference Using Accelerometers on Smartphones (HotMobile).ACM, New York, NY, USA.
[35]
Shijia Pan, Mostafa Mirshekari, Pei Zhang, and Hae Young Noh. 2016. Occupant traffic estimation through structural vibration sensing. (2016).
[36]
Shijia Pan, Ningning Wang, Yuqiu Qian, Irem Velibeyoglu, Hae Young Noh, and Pei Zhang. 2015. Indoor Person Identification Through Footstep Induced Structural Vibration (HotMobile). ACM, New York, NY, USA.
[37]
Pery Pearson. Sound Sampling. http://www.hitl.washington.edu/projects/knowledge_base/virtual-worlds/EVE/I.B.3.a.SoundSampling.html.
[38]
B. Razavi. 2012. Problem of timing mismatch in interleaved ADCs. In Proceedings of the IEEE 2012 Custom Integrated Circuits Conference. 1--8.
[39]
BBC Research. Global Markets and Technologies for Sensors. "http://www.bccresearch.com/market-research/instrumentation-and-sensors/sensors-tech-markets-report-ias006g.html".
[40]
Thomas Schmid, Prabal Dutta, and Mani B. Srivastava. 2010. High-resolution, Low-power Time Synchronization an Oxymoron No More. In Proceedings of the 9th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN '10). ACM.
[41]
Claude E. Shannon. 1949. Communication in the Presence of Noise. Proceedings of the IRE (1949).
[42]
SkyPaw. SkyPaw Multi Measures - Decibel. http://www.skypaw.com/apps/multimeasures/.
[43]
Yunmok Son, Hocheol Shin, Dongkwan Kim, Youngseok Park, Juhwan Noh, Kibum Choi, Jungwoo Choi, and Yongdae Kim. 2015. Rocking Drones with Intentional Sound Noise on Gyroscopic Sensors. In USENIX Security.
[44]
STMicroelectronics. LPY503AL Datasheet. https://www.sparkfun.com/datasheets/Sensors/IMU/lpy503al.pdf.
[45]
Daniel Turner. 2007. Hack: The Nintendo Wii. http://www.technologyreview.com/hack/408183/hack-the-nintendo-wii/.
[46]
Hasan S. Ulusoy, Erol Kalkan, Jon Peter B. Fletcher, Paul Friberg, W. K. Leith, and Krishna Banga. 2012. Design and implementation of a structural health monitoring and alerting system for hospital buildings in the United States. In World Conference on Earthquake Engineering.
[47]
K. J. Van Engen, J. E. Phelps, R. Smiljanic, and B. Chandrasekaran. 2014. Enhancing speech intelligibility: interactions among context, modality, speech style, and masker. Experimental Wireless and the Wireless Engineer (2014).
[48]
WaspMote. Events 2.0 Technical Guide. http://www.libelium.com/downloads/documentation/events-sensor-board_2.0.pdf.
[49]
Anthony Zhang. Speech Recognition. https://github.com/Uberi/speech_recognition.
[50]
Li Zhang, Parth H. Pathak, Muchen Wu, Yixin Zhao, and Prasant Mohapatra. 2015. AccelWord: Energy Efficient Hotword Detection Through Accelerometer (ACM MobiSys).

Cited By

View all
  • (2025)Ambient Light Reflection-Based Eavesdropping Enhanced With cGANIEEE Transactions on Mobile Computing10.1109/TMC.2024.346039224:1(72-85)Online publication date: Jan-2025
  • (2024)Watch the Rhythm: Breaking Privacy with Accelerometer at the Extremely-Low Sampling Rate of 5HzProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3690370(1776-1790)Online publication date: 2-Dec-2024
  • (2024)An Eavesdropping System Based on Magnetic Side-Channel Signals Leaked by SpeakersACM Transactions on Sensor Networks10.1145/363706320:2(1-30)Online publication date: 10-Jan-2024
  • Show More Cited By

Index Terms

  1. Pitchln: eavesdropping via intelligible speech reconstruction using non-acoustic sensor fusion

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    IPSN '17: Proceedings of the 16th ACM/IEEE International Conference on Information Processing in Sensor Networks
    April 2017
    333 pages
    ISBN:9781450348904
    DOI:10.1145/3055031
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 April 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. non-acoustic sensors
    2. privacy
    3. security
    4. sensor fusion
    5. speech reconstruction

    Qualifiers

    • Research-article

    Conference

    IPSN '17
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 143 of 593 submissions, 24%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)44
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 20 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Ambient Light Reflection-Based Eavesdropping Enhanced With cGANIEEE Transactions on Mobile Computing10.1109/TMC.2024.346039224:1(72-85)Online publication date: Jan-2025
    • (2024)Watch the Rhythm: Breaking Privacy with Accelerometer at the Extremely-Low Sampling Rate of 5HzProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3690370(1776-1790)Online publication date: 2-Dec-2024
    • (2024)An Eavesdropping System Based on Magnetic Side-Channel Signals Leaked by SpeakersACM Transactions on Sensor Networks10.1145/363706320:2(1-30)Online publication date: 10-Jan-2024
    • (2024)Accuth+: Accelerometer-Based Anti-Spoofing Voice Authentication on Wrist-Worn WearablesIEEE Transactions on Mobile Computing10.1109/TMC.2023.331483723:5(5571-5588)Online publication date: May-2024
    • (2024)Towards Unconstrained Vocabulary Eavesdropping With mmWave Radar Using GANIEEE Transactions on Mobile Computing10.1109/TMC.2022.322669023:1(941-954)Online publication date: Jan-2024
    • (2024)High-Quality Speech Recovery Through Soundproof Protections via mmWave SensingIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2023.332229521:4(3065-3081)Online publication date: Jul-2024
    • (2024)A Survey of Edge Computing Privacy and Security Threats and Their Countermeasures2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI)10.1109/ISVLSI61997.2024.00093(484-489)Online publication date: 1-Jul-2024
    • (2024)EchoLight: Sound Eavesdropping based on Ambient Light ReflectionIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621338(341-350)Online publication date: 20-May-2024
    • (2024)mmEar: Push the Limit of COTS mmWave Eavesdropping on HeadphonesIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621229(351-360)Online publication date: 20-May-2024
    • (2024)A survey of acoustic eavesdropping attacks: Principle, methods, and progressHigh-Confidence Computing10.1016/j.hcc.2024.1002414:4(100241)Online publication date: Dec-2024
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media