research-article

Open access

MultiFi: Multi Fidelity Interaction with Displays On and Around the Body

Authors:

Matthias Heinisch,

Dieter SchmalstiegAuthors Info & Claims

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Pages 3933 - 3942

https://doi.org/10.1145/2702123.2702331

Published: 18 April 2015 Publication History

Abstract

Display devices on and around the body such as smartwatches, head-mounted displays or tablets enable users to interact on the go. However, diverging input and output fidelities of these devices can lead to interaction seams that can inhibit efficient mobile interaction, when users employ multiple devices at once. We present MultiFi, an interactive system that combines the strengths of multiple displays and overcomes the seams of mobile interaction with widgets distributed over multiple devices. A comparative user study indicates that combined head-mounted display and smartwatch interfaces can outperform interaction with single wearable devices.

Supplementary Material

ZIP File (pn1066.zip)

The auxillary material contains - an extended version of the user study description - detailed statistical test results for the locator and select task presented in the paper - a video demonstrating the individual study conditions

Download
40.71 MB

suppl.mov (pn1066.mp4)

Supplemental video

Download
44.98 MB

MP4 File (p3933.mp4)

Download
154.60 MB

References

[1]

Beaudouin-Lafon, M., Huot, S., Nancel, M., Mackay, W., Pietriga, E., Primet, R., Wagner, J., Chapuis, O., Pillias, C., Eagan, J., Gjerlufsen, T., and Klokmose, C. Multisurface interaction in the wild room. Computer, 45(4), 2012: 48--56.

Digital Library

[2]

Benko, H., Ishak, E. W., and Feiner, S. Crossdimensional gestural interaction techniques for hybrid immersive environments. VR 05. 2005, 209--216.

Digital Library

[3]

Bier, E. A., Stone, M. C., Pier, K., Buxton, W., and DeRose, T. D. Toolglass and magic lenses: the see-through interface. SIGGRAPH. 1993, 73--80.

Digital Library

[4]

Billinghurst, M., and Starner, T. Wearable devices: new ways to manage information. Computer, 32(1), 1999: 57--64.

Digital Library

[5]

Billinghurst, M., Bowskill, J., Dyer, N., and Morphett, J. An evaluation of wearable information spaces. VRAIST 98. 1998, 20--27.

Digital Library

[6]

Billinghurst, M., Grasset, R., and Looser, J. Designing augmented reality interfaces. SIGGRAPH CG, 39(1), 2005: 17--22.

Digital Library

[7]

Boring, S., Baur, D., Butz, A., Gustafson, S., and Baudisch, P. Touch projector: mobile interaction through video. CHI 10. 2010, 2287--2296.

Digital Library

[8]

Budhiraja, R., Lee, G. A., and Billinghurst, M. Using a hhd with a hmd for mobile ar interaction. ISMAR 13. 2013, 1--6.

[9]

Chen, X., Marquardt, N., Tang, A., Boring, S., and Greenberg, S. Extending a mobile device's interaction space through body-centric interaction. MobileHCI 12. 2012, 151--160.

Digital Library

[10]

Chen, X., Grossman, T., Wigdor, D. J., and Fitzmaurice, G. Duet: exploring joint interactions on a smart phone and a smart watch. CHI 14. 2014, 159--168.

Digital Library

[11]

Cockburn, A., Quinn, P., Gutwin, C., Ramos, G., and Looser, J. Air pointing: design and evaluation of spatial target acquisition with and without visual feedback. IJHCS, 69(6), 2011: 401--414.

Digital Library

[12]

Delamare, W., Coutrix, C., and Nigay, L. Designing disambiguation techniques for pointing in the physical world. EICS 13. 2013, 197--206.

Digital Library

[13]

Ens, B., Hincapié-Ramos, J. D., and Irani, P. Ethereal planes: a design framework for 2d information space in 3d mixed reality environments. SUI 14. 2014, 2--12.

Digital Library

[14]

Ens, B., Finnegan, R., and Irani, P. The personal cockpit: a spatial interface for effective task switching on head-worn displays. CHI 14. 2014, 3171--3180.

Digital Library

[15]

Feiner, S., MacIntyre, B., Haupt, M., and Solomon, E. Windows on the world: 2d windows for 3d augmented reality. UIST 93. 1993, 145--155.

Digital Library

[16]

Fitzmaurice, G. W. Situated information spaces and spatially aware palmtop computers. CACM, 36(7), 1993: 39--49.

Digital Library

[17]

Greenberg, S., Marquardt, N., Ballendat, T., Diaz-Marino, R., and Wang, M. Proxemic interactions: the new ubicomp interactions, 18(1), 2011: 42--50.

Digital Library

[18]

Grubert, J., Tuemler, J., Mecke, R., and Schenk, M. Comparative user study of two see-through calibration methods. VR 10. 2010, 269--270.

[19]

Grubert, J., Pahud, M., Grasset, R., Schmalstieg, D., and Seichter, H. The utility of magic lens interfaces on handheld devices for touristic map navigation. PMC, 2014:

[20]

Hart, S. G., and Staveland, L. E. Development of nasa-tlx (task load index): results of empirical and theoretical research. AIP, 52, 1988: 139--183.

[21]

Hassenzahl, M., Burmester, M., and Koller, F. Attrakdi: ein fragebogen zur messung wahrgenommener hedonischer und pragmatischer qualität. In: M&C 03. Springer, 2003, 187--196.

[22]

Hinckley, K., Ramos, G., Guimbretiere, F., Baudisch, P., and Smith, M. Stitching: pen gestures that span multiple displays. AVI 04. 2004, 23--31.

Digital Library

[23]

Lewis, J. R. Psychometric evaluation of an afterscenario questionnaire for computer usability studies: the asq. SIGCHI Bulletin, 23(1), 1991: 78--81.

Digital Library

[24]

Li, F. C. Y., Dearman, D., and Truong, K. N. Virtual shelves: interactions with orientation aware devices. UIST 09. 2009, 125--128.

Digital Library

[25]

Olwal, A., Feiner, S., and Heyman, S. Rubbing and tapping for precise and rapid selection on touch-screen displays. CHI 08. 2008, 295--304.

Digital Library

[26]

Perlin, K., and Fox, D. Pad: an alternative approach to the computer interface. SIGGRAPH 93. 1993, 57--64.

Digital Library

[27]

Rädle, R., Jetter, H.-C., Marquardt, N., Reiterer, H., and Rogers, Y. Huddlelamp: spatially-aware mobile displays for ad-hoc around-the-table collaboration. ITS 14. 2014, 45--54.

Digital Library

[28]

Rashid, U., Nacenta, M. A., and Quigley, A. The cost of display switching: a comparison of mobile, large display and hybrid ui configurations. AVI 12. 2012, 99--106.

Digital Library

[29]

Reichenbacher, T. Adaptive concepts for a mobile cartography. JGS, 11(1), 2001: 43--53.

[30]

Reichlen, B. A. Sparcchair: a one hundred million pixel display. VRAIS 93. 1993, 300--307.

Digital Library

[31]

Rico, J., and Brewster, S. Gestures all around us: user differences in social acceptability perceptions of gesture based interfaces. MobileHCI 09. 2009, 64:1--64:2.

Digital Library

[32]

Vogel, D., and Baudisch, P. Shift: a technique for operating pen-based interfaces using touch. CHI 07. 2007, 657--666.

Digital Library

[33]

Yang, J., and Wigdor, D. Panelrama: enabling easy specification of cross-device web applications. CHI 14. 2014, 2783--2792.

Digital Library

[34]

Zhao, J., Soukoré, R. W., Ren, X., and Balakrishnan, R. A model of scrolling on touch-sensitive displays. IJHCS, 72(12), 2014: 805--821.

Cited By

Daeijavad P(2024)Investigating the Impact of Multiple View Layouts on Users' Visual Task Performance in Extended RealityProceedings of the 2024 International Conference on Advanced Visual Interfaces10.1145/3656650.3656756(1-3)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3656650.3656756
Estalagem NEsteves A(2024)Between Wearable and Spatial Computing: Exploring Four Interaction Techniques at the Intersection of Smartwatches and Head-mounted DisplaysProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3656365(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3656365
Perelman GSerrano MDubois E(2024)Exploiting Physical Referent Features as Input for Multidimensional Data Selection in Augmented RealityACM Transactions on Computer-Human Interaction10.1145/364861331:4(1-40)Online publication date: 19-Sep-2024
https://dl.acm.org/doi/10.1145/3648613
Show More Cited By

Index Terms

MultiFi: Multi Fidelity Interaction with Displays On and Around the Body
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Graphical user interfaces

Recommendations

StretchAR: Exploiting Touch and Stretch as a Method of Interaction for Smart Glasses Using Wearable Straps

Over the past decade, augmented reality (AR) developers have explored a variety of approaches to allow users to interact with the information displayed on smart glasses and head-mounted displays (HMDs). Current interaction modalities such as mid-air ...
Wearables as Context for Guiard-abiding Bimanual Touch
UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and Technology

We explore the contextual details afforded by wearable devices to support multi-user, direct-touch interaction on electronic whiteboards in a way that-unlike previous work-can be fully consistent with natural bimanual-asymmetric interaction as set forth ...
Combining Ring Input with Hand Tracking for Precise, Natural Interaction with Spatial Analytic Interfaces
SUI '16: Proceedings of the 2016 Symposium on Spatial User Interaction

Current wearable interfaces are designed to support short-duration tasks known as micro-interactions. To support productive interfaces for everyday analytic tasks, designers can leverage natural input methods such as direct manipulation and pointing. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

April 2015

4290 pages

ISBN:9781450331456

DOI:10.1145/2702123

General Chairs:
Bo Begole
Huawei, USA
,
Jinwoo Kim
Yonsei University, Korea
,
Program Chairs:
Kori Inkpen
Microsoft Research, USA
,
Woontack Woo
KAIST, Korea

Copyright © 2015 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 April 2015

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

European Commission

Conference

CHI '15

Sponsor:

SIGCHI

CHI '15: CHI Conference on Human Factors in Computing Systems

April 18 - 23, 2015

Seoul, Republic of Korea

Acceptance Rates

CHI '15 Paper Acceptance Rate 486 of 2,120 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

88
Total Citations
View Citations
2,838
Total Downloads

Downloads (Last 12 months)329
Downloads (Last 6 weeks)46

Reflects downloads up to 17 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Daeijavad P(2024)Investigating the Impact of Multiple View Layouts on Users' Visual Task Performance in Extended RealityProceedings of the 2024 International Conference on Advanced Visual Interfaces10.1145/3656650.3656756(1-3)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3656650.3656756
Estalagem NEsteves A(2024)Between Wearable and Spatial Computing: Exploring Four Interaction Techniques at the Intersection of Smartwatches and Head-mounted DisplaysProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3656365(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3656365
Perelman GSerrano MDubois E(2024)Exploiting Physical Referent Features as Input for Multidimensional Data Selection in Augmented RealityACM Transactions on Computer-Human Interaction10.1145/364861331:4(1-40)Online publication date: 19-Sep-2024
https://dl.acm.org/doi/10.1145/3648613
Gonzalez EChatterjee IGonzalez-Franco MColaço AAhuja K(2024)Intent-driven input device arbitration for XRExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650758(1-5)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650758
Takeda KManabe H(2024)Screen Augmentation Technique Using AR Glasses and Smartphone without External SensorsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3648664(1-5)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3648664
Manakhov PSidenmark LPfeuffer KGellersen H(2024)Gaze on the Go: Effect of Spatial Reference Frame on Visual Target Acquisition During Physical Locomotion in Extended RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642915(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642915
Zhu FSousa MSidenmark LGrossman T(2024)PhoneInVR: An Evaluation of Spatial Anchoring and Interaction Techniques for Smartphone Usage in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642582(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642582
Wentzel JAnderson FFitzmaurice GGrossman TVogel D(2024)SwitchSpace: Understanding Context-Aware Peeking Between VR and Desktop InterfacesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642358(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642358
Lee BSedlmair MSchmalstieg D(2024)Design Patterns for Situated Visualization in Augmented RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332739830:1(1324-1335)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3327398
Grubert JWitzani LOtte AGesslein TKranz MKristensson P(2024)Text Entry Performance and Situation Awareness of a Joint Optical See-Through Head-Mounted Display and Smartphone SystemIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.330931630:8(5830-5846)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3309316
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents