research-article

Reality-based interaction: a framework for post-WIMP interfaces

Authors:

Robert J.K. Jacob,

Audrey Girouard,

Leanne M. Hirshfield,

Michael S. Horn,

Erin Treacy Solovey,

Jamie ZigelbaumAuthors Info & Claims

CHI '08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 201 - 210

https://doi.org/10.1145/1357054.1357089

Published: 06 April 2008 Publication History

Abstract

We are in the midst of an explosion of emerging human-computer interaction techniques that redefine our understanding of both computers and interaction. We propose the notion of Reality-Based Interaction (RBI) as a unifying concept that ties together a large subset of these emerging interaction styles. Based on this concept of RBI, we provide a framework that can be used to understand, compare, and relate current paths of recent HCI research as well as to analyze specific interaction designs. We believe that viewing interaction through the lens of RBI provides insights for design and uncovers gaps or opportunities for future research.

References

[1]

Abowd, G.D. and Dix, A.J., Integrating Status and Event Phenomena in Formal Specifications of Interactive Systems. in Proc. SIGSOFT 1994, (1994), Addison-Wesley/ACM Press.

Digital Library

[2]

Agarawala, A. and Balakrishnan, R. Keepin' it real: pushing the desktop metaphor with physics, piles and the pen Proc. CHI2006, ACM Press, 2006.

Digital Library

[3]

Ängeslevä, J., Oakley, I., Hughes, S. and O'Modhrain, S., Body Mnemonics Portable device interaction design concept. in UIST, (2003).

[4]

Apted, T., Kay, J. and Quigley, A. Tabletop sharing of digital photographs for the elderly Proc. of SIGCHI, ACM Press, 2006.

Digital Library

[5]

Bae, S.-H., Kobayash, T., Kijima, R. and Kim, W.-S., Tangible NURBS-curve manipulation techniques using graspable handles on a large display. in Proc. UIST 2004, (2004), ACM Press, 81--90.

Digital Library

[6]

Beaudouin-Lafon, M. Instrumental Interaction: An Interaction Model for Designing Post-WIMP User Interfaces Proc. CHI 2000, Addison-Wesley/ACM Press, 2000, 446--453.

Digital Library

[7]

Beeharee, A. and Steed, A. Exploiting real world knowledge in ubiquitous applications. Personal and Ubiquitous Computing, 11. 429--437.

Digital Library

[8]

Bellotti, V., Back, M., Edwards, W.K., Grinter, R.E., Henderson, A. and Lopes, C. Making Sense of Sensing Systems: Five Questions for Designers and Researchers Proc. CHI 2002, ACM Press, 2002, 415--422.

Digital Library

[9]

Benford, S., Schnadelbach, H., Koleva, B., Anastasi, R., Greenhalgh, C., Rodden, T., Green, J., Ghali, A., Pridmore, T., Gaver, B., Boucher, A., Walker, B., Pennington, S., Schmidt, A., Gellersen, H. and Steed, A. Expected, sensed, and desired: A framework for designing sensing--based interaction. ACM Transactions Computer-Human Interaction, 12 (1). 3--30.

Digital Library

[10]

Bowman, D.A., Kruijff, E., LaViola, J.J. and Poupyrev, I. 3D User Interfaces: Theory and Practice. Addison-Wesley/Pearson Education, 2004.

Digital Library

[11]

Buur, J., Jensen, M.V. and Djajadiningrat, T., Please touch tangible UIs: Hands-only scenarios and video action walls: novel methods for tangible user interaction design. in Proc. DIS'04, (2004).

Digital Library

[12]

Buxton, B. http://www.billbuxton.com/multitouchOverview.html, 2007.

[13]

Christou, G. Towards a new method for the evaluation of reality based interaction CHI '07 extended abstracts, ACM, San Jose, CA, USA, 2007.

Digital Library

[14]

Coutrix, C. and Nigay, L. Mixed reality: a model of mixed interaction. Proceedings of the working conference on Advanced visual interfaces.

Digital Library

[15]

Dickie, C., Vertegaal, R., Sohn, C. and Cheng, D. eyeLook: using attention to facilitate mobile media consumption Proc. UIST 2005, ACM Press, 2005.

Digital Library

[16]

Dourish, P. Where The Action Is: The Foundations of Embodied Interaction, MIT Press, Cambridge, Mass., 2001.

[17]

Dubois, E. and Gray, P. A Design-Oriented Information-Flow Refinement of the ASUR Interaction Model. Proceedings of EHCI 2007.

[18]

Fishkin, K.P. A Taxonomy for and Analysis of Tangible Interfaces. Personal and Ubiquitous Computing, 8 (5). 347 -- 358.

Digital Library

[19]

Fishkin, K.P., Moran, T.P. and Harrison, B.L. Embodied User Interfaces: Toward Invisible User Interfaces Proc. of EHCI'98, 1998.

Digital Library

[20]

Forlines, C. and Shen, C., Touch: DTLens: multi-user tabletop spatial data exploration. in Proc. UIST 2005, (2005).

Digital Library

[21]

Hayes, P.J. The second naive physics manifesto Cognitive Science Technical Report URCS-10, University of Rochester, 1983.

[22]

Hornecker, E. and Buur, J., Getting a Grip on Tangible Interaction: A Framework on Physical Space and Social Interaction. in Proc. of CHI 2006, (2006), ACM, 437--446.

Digital Library

[23]

Hurtienne, J. and Israel, J.H. Image schemas and their metaphorical extensions: intuitive patterns for tangible interaction Proc. TEI 2007, ACM Press, 2007.

Digital Library

[24]

Hutchins, E.L., Hollan, J.D. and Norman, D.A. Direct Manipulation Interfaces. Draper, D.A.N.a.S.W. ed. User Centered System Design: New Perspectives on Human-computer Interaction, Lawrence Erlbaum, Hillsdale, N.J., 1986, 87--124.

[25]

Jackson, R.L. and Fagan, E., Collaboration and learning within immersive virtual reality. in Proc. Collaborative Virtual Environments, (2000), ACM Press, 83 -- 92.

Digital Library

[26]

Jacob, R.J.K., Girouard, A., Hirshfield, L.M., Horn, M.S., Shaer, O., Solovey, E.T. and Zigelbaum, J., Reality-Based Interaction: Unifying the New Generation of Interaction Styles (Work in Progress paper). in CHI 2007, (2007), 2465--2470.

Digital Library

[27]

Jacob, R.J.K., Girouard, A., Horn, M., Miller, L., Shaer, O., Treacy, E. and Zigelbaum, J. What Is the Next Generation of Human--Computer Interaction? interactions, 2007.

Digital Library

[28]

Kalanithi, J. Connectibles: Tangible Social Networking, MS Thesis, MIT Media Lab, Cambridge, 2007, 132pp.

[29]

Klemmer, S.R., Hartmann, B. and Takayama, L., How bodies matter: five themes for interaction design. in Proc. DIS 2006, (2006), ACM Press, 140--149.

Digital Library

[30]

MacLean, A., Young, R.M., Bellotti, V.M.E. and Moran, T.P. Questions, Options, and Criteria: Elements of Design Space Analysis. Human-Computer Interaction, 6. 201--250.

Digital Library

[31]

Maeda, J. The Laws of Simplicity (Simplicity: Design, Technology, Business, Life). The MIT Press, 2006.

Digital Library

[32]

Merrill, D. and Maes, P., Augmenting Looking, Pointing and Reaching Gestures to Enhance the Searching and Browsing of Physical Objects. in Proc. Pervasive'07, (2007).

Digital Library

[33]

Milgram, P. and Kishino, F., Augmented reality: A class of displays on the reality-virtuality continuum. in SPIE, (1994), 282--292.

[34]

Mohler, B.J., Thompson, W.B., Creem-Regehr, S.H., Willemsen, P., Herbert L. Pick, Jr. and Rieser, J.J. Calibration of locomotion resulting from visual motion in a treadmill-based virtual environment. ACM Trans. Appl. Percept., 4 (1). 4.

Digital Library

[35]

Nielsen, J. Noncommand User Interfaces Comm. ACM, 1993, 83--99.

Digital Library

[36]

Poupyrev, I., Newton--Dunn, H. and Bau, O. D20: interaction with multifaceted display devices CHI '06 extended abstracts, ACM Press, 2006.

Digital Library

[37]

Rohrer, T. Metaphors We Compute By: Bringing Magic into Interface Design, Center for the Cognitive Science of Metaphor, Philosophy Department, University of Oregon, 1995.

[38]

Second Life. http://secondlife.com/.

[39]

Shneiderman, B. Direct Manipulation. A Step Beyond Programming Languages. IEEE Transactions on Computers, 16 (8). 57.

Digital Library

[40]

Shneiderman, B. Leonardo's Laptop: Human Needs and the New Computing Technologies, MIT Press, Cambridge, Mass., 2002.

Digital Library

[41]

Shneiderman, B. Why Not Make Interfaces Better Than 3D Reality? IEEE Computer Graphics and Applications, 2003, 12--15.

Digital Library

[42]

Slater, M., Usoh, M. and Steed, A. Taking steps: the influence of a walking technique on presence in virtual reality. ACM Trans. Comput.-Hum. Interact., 2 (3). 201--219.

Digital Library

[43]

Smith, J.D., Vertegaal, R. and Sohn, C., ViewPointer: lightweight calibration-free eye tracking for ubiquitous handsfree deixis. in Proc. UIST 2005, (2005).

Digital Library

[44]

Streitz, N.A., Tandler, P., Muller-Tomfelde, C. and Konomi, S. Roomware: Toward the Next Generation of Human-Computer Interaction Based on an Integrated Design of Real and Virtual Worlds. Carroll, J.M. ed. Human-Computer Interaction in the New Millenium, Addison-Wesley/ACM Press, Reading, Mass., 2001.

[45]

Tanriverdi, V. A Virtual Reality Interface Design (VRID) Model and Methodology, Tufts University, 2001.

[46]

Tanriverdi, V. and Jacob, R.J.K., Interacting with Eye Movements in Virtual Environments. in Proc. CHI 2000, (2000), Addison-Wesley/ACM Press, 265--272.

Digital Library

[47]

Ullmer, B. and Ishii, H. Emerging Frameworks for Tangible User Interfaces. Carroll, J.M. ed. Human-Computer Interaction in the New Millenium, Addison-Wesley/ACM Press, Reading, Mass., 2001.

[48]

Ullmer, B., Ishii, H. and Jacob, R.J.K. Token+Constraint Systems for Tangible Interaction with Digital Information ACM TOCHI, 2005, 81--118.

Digital Library

[49]

Underkoffler, J. and Ishii, H. Urp: A Luminous-Tangible Workbench for Urban Planning and Design Proc. CHI'99, Addison-Wesley/ACM Press, 1999, 386--393.

Digital Library

[50]

Usoh, M., Arthur, K., Whitton, M.C., Bastos, R., Steed, A., Slater, M. and Brooks, F.P. Walking > walking-in-place > flying, in virtual environments Proc. SIGGRAPH'99, Addison-Wesley/ACM Press, 1999, 359--364.

Digital Library

[51]

Van Dam, A. Post-WIMP user interfaces. Commun. ACM, 40 (2). 63--67.

Digital Library

[52]

Vinson, N.G. Design guidelines for landmarks to support navigation in virtual environments Proc CHI, ACM Press, 1999, 278--285.

Digital Library

[53]

Vogel, D. and Balakrishnan, R., Distant freehand pointing and clicking on very large, high resolution displays. in Proc. UIST 2005, (2005).

Digital Library

[54]

Warren, J. A Wii Workout: When Videogames Hurt Wall Street Journal Online, 2006.

[55]

Weiser, M. The Computer for the Twenty-first Century Scientific American, 1991, 94--104.

[56]

Zigelbaum, J., Horn, M., Shaer, O. and Jacob, R., The tangible video editor: collaborative video editing with active tokens. in Proc. TEI '07, (2007), ACM, 43--46.

Digital Library

Cited By

Yeo AKwok BJoshna AChen KLee J(2024)Entering the Next Dimension: A Review of 3D User Interfaces for Virtual RealityElectronics10.3390/electronics1303060013:3(600)Online publication date: 1-Feb-2024
https://doi.org/10.3390/electronics13030600
Herder EStojko LStrecker JNeumayr TYigitbas EAugstein M(2024)Towards new realities: implications of personalized online layers in our daily livesi-com10.1515/icom-2024-001723:2(221-229)Online publication date: 18-Jun-2024
https://doi.org/10.1515/icom-2024-0017
Shapiro KQuijada Leyton PStemple LUberti D(2024)Interacting from Afar: A Study of the Relationship Between Usability and Presence in Object Selection at a DistanceProceedings of 21st ACM SIGGRAPH Conference on Visual Media Production10.1145/3697294.3697305(1-9)Online publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1145/3697294.3697305
Show More Cited By

Index Terms

Reality-based interaction: a framework for post-WIMP interfaces
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Instrumental interaction: an interaction model for designing post-WIMP user interfaces
CHI '00: Proceedings of the SIGCHI conference on Human Factors in Computing Systems

This article introduces a new interaction model called Instrumental Interaction that extends and generalizes the principles of direct manipulation. It covers existing interaction styles, including traditional WIMP interfaces, as well as new interaction ...
Reality-based interaction: unifying the new generation of interaction styles
CHI EA '07: CHI '07 Extended Abstracts on Human Factors in Computing Systems

We are in the midst of an explosion of emerging human-computer interaction techniques that have redefined our understanding of both computers and interaction. We propose the notion of Reality-Based Interaction (RBI) as a unifying concept that ties ...
Body-centric interaction techniques for very large wall displays
NordiCHI '10: Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries

We examine the design space of interaction techniques for very large wall displays by drawing from existing theory and practice for reality-based interfaces and whole-body interfaces. We also apply insights drawn from research in psychology about the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2008

1870 pages

ISBN:9781605580111

DOI:10.1145/1357054

General Chairs:
Mary Czerwinski
Microsoft Research, USA
,
Arnie Lund
Microsoft, USA
,
Program Chair:
Desney Tan
Microsoft Research, USA

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 April 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '08

Sponsor:

CHI '08: CHI Conference on Human Factors in Computing Systems

April 5 - 10, 2008

Florence, Italy

Acceptance Rates

CHI '08 Paper Acceptance Rate 157 of 714 submissions, 22%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

533
Total Citations
View Citations
8,207
Total Downloads

Downloads (Last 12 months)535
Downloads (Last 6 weeks)48

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yeo AKwok BJoshna AChen KLee J(2024)Entering the Next Dimension: A Review of 3D User Interfaces for Virtual RealityElectronics10.3390/electronics1303060013:3(600)Online publication date: 1-Feb-2024
https://doi.org/10.3390/electronics13030600
Herder EStojko LStrecker JNeumayr TYigitbas EAugstein M(2024)Towards new realities: implications of personalized online layers in our daily livesi-com10.1515/icom-2024-001723:2(221-229)Online publication date: 18-Jun-2024
https://doi.org/10.1515/icom-2024-0017
Shapiro KQuijada Leyton PStemple LUberti D(2024)Interacting from Afar: A Study of the Relationship Between Usability and Presence in Object Selection at a DistanceProceedings of 21st ACM SIGGRAPH Conference on Visual Media Production10.1145/3697294.3697305(1-9)Online publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1145/3697294.3697305
Lee LLin Z(2024)Danger, Nuisance, Disregard: Analyzing User-Generated Videos for Augmented Reality Gameplay on Hand-held DevicesProceedings of the ACM on Human-Computer Interaction10.1145/36770638:CHI PLAY(1-33)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677063
Artizzu VLuyten KRuiz GSpano L(2024)ViRgilites: Multilevel Feedforward for Multimodal Interaction in VRProceedings of the ACM on Human-Computer Interaction10.1145/36586458:EICS(1-24)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3658645
Ferron AHachet MJansen Y(2024)Visualizing the Environmental Impact of Dietary Choices: Exploring an Interactive Mini-World as a Proxy to Communicate Three Planetary Boundaries: Visualiser l’impact environnemental des choix alimentaires : exploration d’un mini-monde interactif comme visualisation mandataire pour communiquer trois limites planétairesProceedings of the 35th Conference on l'Interaction Humain-Machine10.1145/3649792.3649804(1-16)Online publication date: 25-Mar-2024
https://dl.acm.org/doi/10.1145/3649792.3649804
Law MNwagwu NKwatra ALee SDiangelis DYu NGonzalez-Pumariega GRajesh AHoffman G(2024)Affinity Diagramming with a RobotACM Transactions on Human-Robot Interaction10.1145/364151413:1(1-41)Online publication date: 31-Jan-2024
https://dl.acm.org/doi/10.1145/3641514
Assor AProuzeau ADragicevic PHachet M(2024)Augmented Reality Waste Accumulation VisualizationsACM Journal on Computing and Sustainable Societies10.1145/36369702:2(1-29)Online publication date: 13-Jan-2024
https://dl.acm.org/doi/10.1145/3636970
de Kreij SRanasinghe CDegbelo A(2024)Data Physicalization and Tangible Manipulation for Engaging Children with Data: An Example with Air Quality DataProceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3655788(507-516)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3628516.3655788
Echeverri DCiolfi LHogan TDöring TJenkins Tvan Dijk JHuron SLi ZCoyle DSigner B(2024)Puppetry in Tangible Narratives: Interactive and Collaborative Storytelling in The Non-myth of the Noble RedProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3635330(1-4)Online publication date: 11-Feb-2024
https://dl.acm.org/doi/10.1145/3623509.3635330
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents