research-article

Acoustruments: Passive, Acoustically-Driven, Interactive Controls for Handheld Devices

Authors:

Eric Brockmeyer,

Scott E. Hudson,

Chris HarrisonAuthors Info & Claims

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Pages 2161 - 2170

https://doi.org/10.1145/2702123.2702414

Published: 18 April 2015 Publication History

Abstract

We introduce Acoustruments: low-cost, passive, and power-less mechanisms, made from plastic, that can bring rich, tangible functionality to handheld devices. Through a structured exploration, we identified an expansive vocabulary of design primitives, providing building blocks for the construction of tangible interfaces utilizing smartphones' existing audio functionality. By combining design primitives, familiar physical mechanisms can all be constructed from passive elements. On top of these, we can create end-user applications with rich, tangible interactive functionalities. Our experiments show that Acoustruments can achieve 99% accuracy with minimal training, is robust to noise, and can be rapidly prototyped. Acoustruments adds a new method to the toolbox HCI practitioners and researchers can draw upon, while introducing a cheap and passive method for adding interactive controls to consumer products.

Supplementary Material

suppl.mov (pn1533-file3.mp4)

Supplemental video

Download
87.20 MB

MP4 File (p2161-laput.mp4)

Download
129.77 MB

References

[1]

Aumi, M.T.I., Gupta, S., Goel, M., Larson, E., and Patel, S. DopLink: using the doppler effect for multidevice interaction. In Proc. UbiComp '13.

Digital Library

[2]

Avrahami, D. and Hudson, S.E. Forming interactivity: a tool for rapid prototyping of physical interactive products. In Proc. DIS '02.

Digital Library

[3]

Baudisch, P., Becker, T., and Rudeck, F. Lumino": Tangible Blocks for Tabletop Computers Based on Glass Fiber Bundles. In Proc. CHI '10.

Digital Library

[4]

Benade, A.H. Fundamentals of musical acoustics. Oxford Press, 1976.

[5]

Brandt, A. Noise and Vibration Analysis. Wiley, 2011.

[6]

Briggs, G.A. Musical instruments and audio. Wharfedale Wireless Works, Idle; Yorkshire, 1965.

[7]

Brockmeyer, E., Poupyrev, I., and Hudson, S. PAPILLON: designing curved display surfaces with printed optics. In Proc. ACM UIST '13, 457--462.

Digital Library

[8]

Carse, A. Musical wind instruments. Da Capo Press, '75.

[9]

Chan, L., Müller, S., Roudaut, A., and Baudisch, P. CapStones and ZebraWidgets: sensing stacks of building blocks, dials and sliders on capacitive touch screens. In Proc. UIST '12.

[10]

Crevoisier, A., Polotti, P., and Politecnico, I. Tangible Acoustic Interfaces and their Applications for the Design of New Musical Instruments. In Proc NIME '05.

Digital Library

[11]

Culver, C.A. Musical acoustics. McGraw-Hill, 1956.

[12]

Daubeny, U. Orchestral wind instruments, ancient and modern. Books for Libraries Press, 1970.

[13]

Debost, M. The simple flute!: from A to Z. Oxford University Press, Oxford; New York, 2002.

[14]

Fletcher, N.H. and Rossing, T.D. The physics of musical instruments. Springer, New York, 1998.

[15]

Gloth, G. and Sinapius, M. Analysis of swept-sine runs during modal identification. In Proc. Mech. Syst. Signal Process. 18, (2004), 1421--1441.

[16]

Greenberg, S. and Fitchett, C. Phidgets: easy development of physical interfaces through physical widgets. In Proc. UIST '01, (2001), 209--218.

Digital Library

[17]

Gupta, S., Morris, D., Patel, S., and Tan, D. SoundWave: Using the Doppler Effect to Sense Gestures. In Proc. ACM UIST '12, (2012), 1911--1914.

Digital Library

[18]

Harrison, C. and Hudson, S.E. Scratch input: creating large, inexpensive, unpowered and mobile finger input surfaces. In Proc. ACM UIST '12, (2008), 205--208.

Digital Library

[19]

Harrison, C., Tan, D., and Morris, D. Skinput: appropriating the body as an input surface. In Proc. CHI '10, ACM Press (2010), 453.

Digital Library

[20]

Von Helmholtz, H. and Ellis, A.J. On the sensations of tone as a physiological basis for the theory of music. Dover Publications, 1954.

[21]

Hirschberg, A., Kergomard, J., and Weinreich, G. Mechanics of musical instruments. Springer-Verlag, Wien; New York, 1995.

[22]

Hwang, S., Ahn, M., and Wohn, K. MagGetz: Customizable Passive Tangible Controllers on and Around Conventional Mobile Devices. In Proc. UIST'13.

Digital Library

[23]

Ishii, H., Wisneski, C., Orbanes, J., Chun, B., and Paradiso, J. PingPongPlus: design of an athletic-tangible interface for computer-supported cooperative play. In Proc. UIST '07.

[24]

Kratz, S., Westermann, T., Rohs, M., and Essl, G. CapWidgets: tangile widgets versus multi-touch controls on mobile devices. In Proc. CHI 2011, 1351--1356.

Digital Library

[25]

Kuo, Y.-S., Verma, S., Schmid, T., and Dutta, P. Hijacking power and bandwidth from the mobile phone's audio interface. In Proc. ACM DEV '10.

Digital Library

[26]

Liang, R., Chan, L., and Kuo, H.T.H. GaussBricks: Magnetic Building Blocks for Constructive Tangible Interactions on Portable Displays. In Proc. CHI'14.

Digital Library

[27]

Mueller, S., Kruck, B., and Baudisch, P. LaserOrigami: laser-cutting 3D objects. In Proc. CHI '13, 2585.

Digital Library

[28]

Mueller, S., Mohr, T., Guenther, K., Frohnhofen, J., and Baudisch, P. faBrickation": Fast 3D Printing of Functional Objects by Integrating Construction Kit Building Blocks. In Proc. CHI '14.

Digital Library

[29]

Mujibiya, A., Cao, X., Tan, D.S., Morris, D., Patel, S.N., and Rekimoto, J. The sound of touch: on-body touch and gesture sensing based on transdermal ultrasound propagation. In Proc. ITS '13.

Digital Library

[30]

Munjal, M.L. Acoustics of ducts and mufflers with application to exhaust and ventilation system design. Wiley, New York, 1987.

[31]

Ono, M., Shizuki, B., and Tanaka, J. Touch & activate: adding interactivity to existing objects using active acoustic sensing. In Proc. ACM UIST '13.

Digital Library

[32]

Patel, S.N. and Abowd, G.D. Blui: low-cost localized blowable user interfaces. In Proc. UIST '07.

Digital Library

[33]

Priyantha, N.B., Chakraborty, A., & Balakrishnan, H. Cricket location-support system. In Proc. MobiCom '00.

Digital Library

[34]

Raj, B., Kalgaonkar, K., Harrison, C., and Dietz, P. Ultrasonic Doppler Sensing in HCI. In. Pervasive'12.

Digital Library

[35]

Robinson, A., Verma, S., and Dutta, P. AudioDAQ: turning the mobile phone's headset port into a universal data acquisition interface. In IPSN '12.

Digital Library

[36]

Sato, M., Poupyrev, I., and Harrison, C. Touché: enhancing touch interaction on humans, screens, liquids, and everyday objects. In Proc. CHI '12.

Digital Library

[37]

Savage, V., Chang, C., and Hartmann, B. Sauron: embedded single-camera sensing of printed physical user interfaces. In Proc. ACM UIST '13.

Digital Library

[38]

Savage, V., Schmidt, R., Grossman, T., Fitzmaurice, G., and Hartmann, B. A Series of Tubes: Adding Interactivity to 3D Prints Using Internal Pipes. In Proc. UIST '14.

Digital Library

[39]

Savage, V., Zhang, X., and Hartmann, B. Midas: Fabricating Custom Capacitive Touch Sensors to Prototype Interactive Objects. In Proc. ACM UIST '12.

Digital Library

[40]

Sinha, D., Springer, K., Han, W., Lizon, D., and Houlton, R. Swept-frequency acoustic interferometry technique for noninvasive chemical diagnostics. In Proc. 3rd Int. Conf. On-Site Analysis (1995).

[41]

Takemura, K., Ito, A., Takamatsu, J., and Ogasawara, T. Active bone-conducted sound sensing for wearable interfaces. In Proc. UIST '11.

Digital Library

[42]

Tarzia, S.P., Dick, R.P., Dinda, P.A., and Memik, G. Sonar-based measurement of user presence and attention. In Proc. Ubicomp '09.

Digital Library

[43]

Tromlitz, J.G. and Powell, A. The keyed flute. Clarendon Press"; Oxford University Press, 1996.

[44]

Watanabe, C., Cassinelli, A., Watanabe, Y., and Ishikawa, M. Generic method for crafting deformable interfaces to physically augment smartphones. CHI EA'14.

Digital Library

[45]

Watanabe, H., Terada, T., and Tsukamoto, M. Ultrasound-based movement sensing, gesture-, and context-recognition. In Proc. ISWC '13.

Digital Library

[46]

Weiss, M., Wagner, J., Jansen, Y., et al. SLAP Widgets: Bridging the Gap Between Virtual and Physical Controls on Tabletops. In Proc. CHI '14.

Digital Library

[47]

Willis, K., Brockmeyer, E., Hudson, S., and Poupyrev, I. Printed optics: 3D printing of embedded optical elements for interactive devices. In Proc. UIST '12.

Digital Library

[48]

Romo Robot. http://www.romotive.com/.

[49]

Thingiverse. http://www.thingiverse.com/.

[50]

The First 3D-Printed Sax. http://gizmodo.com/the-first3d-printed-saxophone-sounds-surprisingly-dece1616272037.

Cited By

Palma GPourjafarian NSteimle JCignoni P(2024)Capacitive Touch Sensing on General 3D SurfacesACM Transactions on Graphics10.1145/365818543:4(1-20)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658185
Nolan JQian KZhang X(2024)KeyStubProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314427:4(1-23)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631442
Getschmann CEchtler FCiolfi LHogan TDöring TJenkins Tvan Dijk JHuron SLi ZCoyle DSigner B(2024)LensLeech: On-Lens Interaction for Arbitrary Camera DevicesProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3633382(1-10)Online publication date: 11-Feb-2024
https://dl.acm.org/doi/10.1145/3623509.3633382
Show More Cited By

Index Terms

Acoustruments: Passive, Acoustically-Driven, Interactive Controls for Handheld Devices
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Acoustruments: passive, acoustically-driven, interactive controls for handheld devices
SIGGRAPH '15: ACM SIGGRAPH 2015 Emerging Technologies

Smartphones and handheld devices are increasingly being used in interactive applications beyond their conventional touchscreens. For example, tangibles allow users to interact with mobile devices using physical objects both on screen and around the ...
Expressive touch: studying tapping force on tabletops
CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

This paper investigates users' ability to perform force-sensitive tapping and explores its potential as an input modality in touch-based systems. We study force-sensitive tapping using Expressive Touch, a tabletop interface that infers tapping force ...
A ring-shaped interactive device for large remote display and mobile device control
UbiComp '11: Proceedings of the 13th international conference on Ubiquitous computing

In this demonstration, a novel human-computer interaction device is proposed to realize finger touching for large display and mobile device control, without a touchscreen or a touch pad. In this method, interaction commands are input in a same way as ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

April 2015

4290 pages

ISBN:9781450331456

DOI:10.1145/2702123

General Chairs:
Bo Begole
Huawei, USA
,
Jinwoo Kim
Yonsei University, Korea
,
Program Chairs:
Kori Inkpen
Microsoft Research, USA
,
Woontack Woo
KAIST, Korea

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 April 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '15

Sponsor:

SIGCHI

CHI '15: CHI Conference on Human Factors in Computing Systems

April 18 - 23, 2015

Seoul, Republic of Korea

Acceptance Rates

CHI '15 Paper Acceptance Rate 486 of 2,120 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

79
Total Citations
View Citations
1,484
Total Downloads

Downloads (Last 12 months)125
Downloads (Last 6 weeks)12

Reflects downloads up to 19 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Palma GPourjafarian NSteimle JCignoni P(2024)Capacitive Touch Sensing on General 3D SurfacesACM Transactions on Graphics10.1145/365818543:4(1-20)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658185
Nolan JQian KZhang X(2024)KeyStubProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314427:4(1-23)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631442
Getschmann CEchtler FCiolfi LHogan TDöring TJenkins Tvan Dijk JHuron SLi ZCoyle DSigner B(2024)LensLeech: On-Lens Interaction for Arbitrary Camera DevicesProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3633382(1-10)Online publication date: 11-Feb-2024
https://dl.acm.org/doi/10.1145/3623509.3633382
Lee CZhang RAgarwal DYu TGunda VLopez OKim JYin SDong BLi KSakashita MGuimbretiere FZhang C(2024)EchoWrist: Continuous Hand Pose Tracking and Hand-Object Interaction Recognition Using Low-Power Active Acoustic Sensing On a WristbandProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642910(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642910
Campos Zamora DDogan MSiu AKoh EXiao C(2024)MoiréWidgets: High-Precision, Passive Tangible Interfaces via Moiré EffectProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642734(1-10)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642734
Ye QYong ZHan BYen CZheng C(2024)PaperTouch: Tangible Interfaces through Paper Craft and Touchscreen DevicesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642571(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642571
Ye ZWang YChen Y(2024)CasePad: Privacy-preserving Finger Activity Sensing via Passive Acoustic Signals Enhanced by Mini-Structures in Smartphone Cases2024 33rd International Conference on Computer Communications and Networks (ICCCN)10.1109/ICCCN61486.2024.10637508(1-9)Online publication date: 29-Jul-2024
https://doi.org/10.1109/ICCCN61486.2024.10637508
Gyory PBae SYang RDo EZheng C(2023)Marking Material Interactions with Computer VisionProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580643(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580643
Watanabe HSasaki KTerada TTsukamoto M(2022)Method for Recognizing Pressing Position and Shear Force Using Active Acoustic Sensing on Gel PlatesSensors10.3390/s2224995122:24(9951)Online publication date: 16-Dec-2022
https://doi.org/10.3390/s22249951
Gyory PMaiden NSas C(2022)Creating Platforms to Support Craft and Creativity in Game Controller DesignProceedings of the 14th Conference on Creativity and Cognition10.1145/3527927.3533733(708-710)Online publication date: 20-Jun-2022
https://dl.acm.org/doi/10.1145/3527927.3533733
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents