article

Expected, sensed, and desired: A framework for designing sensing-based interaction

Authors:

Holger Schnädelbach,

Boriana Koleva,

Chris Greenhalgh,

Jonathan Green,

Brendan Walker,

Sarah Pennington,

Albrecht Schmidt,

Hans Gellersen,

Anthony SteedAuthors Info & Claims

ACM Transactions on Computer-Human Interaction (TOCHI), Volume 12, Issue 1

Pages 3 - 30

https://doi.org/10.1145/1057237.1057239

Published: 01 March 2005 Publication History

Abstract

Movements of interfaces can be analyzed in terms of whether they are expected, sensed, and desired. Expected movements are those that users naturally perform; sensed are those that can be measured by a computer; and desired movements are those that are required by a given application. We show how a systematic comparison of expected, sensed, and desired movements, especially with regard to how they do not precisely overlap, can reveal potential problems with an interface and also inspire new features. We describe how this approach has been applied to the design of three interfaces: pointing flashlights at walls and posters in order to play sounds; the Augurscope II, a mobile augmented reality interface for outdoors; and the Drift Table, an item of furniture that uses load sensing to control the display of aerial photographs. We propose that this approach can help to build a bridge between the analytic and inspirational approaches to design and can help designers meet the challenges raised by a diversification of sensing technologies and interface forms, increased mobility, and an emerging focus on technologies for everyday life.

References

[1]

Balakrishnan, R., Baudel, T., Kurtenbach, G., and Fitzmaurice, G. 1997. The Rockin'Mouse: Integral 3D manipulation on a plane. In Proceedings of the Conference on Human Factors in Computing Systems. Atlanta, GA. 311--318.

[2]

Bellotti, V., Back, M., Edwards, W. K., Grinter, R. E., Henderson, A., and Lopes, C. 2002. Making sense of sensing systems: Five questions for designers and researchers. In Conference on Human Factors in Computing Systems. Minneapolis, MN. 415--422.

[3]

Bleser, T. W. and Sibert, J. L. 1990. Toto: A tool for selecting interaction techniques. In Proceedings of the 3rd annual ACM SIGGRAPH Symposium on User interface Software and Technology. Snowbird, UT. 135--142.

[4]

Boucher, A., Gaver, W., Pennington, S., and Walker, B. 2003. Workbook one: Ideas, scenarios and proposals for the home, available at www.interaction.rca.ac.uk/equator/papers/workbook1.pdf.

[5]

Bowers, J. and Hellstrom, S. 2000. Simple interfaces to complex sound in improvised electronic music. In Proceedings of the Conference on Human Factors in Computing Systems, Supplementary Proceedings. The Hague, Netherlands. 125--126.

[6]

Bowers, J. and Pycock, J. 1994. Talking through design: Requirements and resistance in cooperative prototyping. In Proceedings of the Conference on Human Factors in Computing Systems, Supplementary Proceedings. New York, NY. 299--305.

[7]

Buxton, W. 1983. Lexical and pragmatic considerations of input structures. Comput. Graph. 17, 1, 31--37.

[8]

Card, S. K., Mackinlay, J. D., and Robertson, G. G. 1991. A morphological analysis of the design space of input devices. ACM Trans. Inform. Sys. 9, 2, 99--122.

[9]

Cheverst, K., Davies, N., Mitchell, K., Friday, A., and Efstratiou C. 2000. Developing a context-aware electronic tourist guide: Some issues and experiences. In Proceedings of the Conference on Human Factors in Computing Systems. The Hague, Netherlands. 17--24.

[10]

Cruz-Neira, C., Sandin, D. J., Defant, T. A., Kenyon, R. V., and Hart, J. C. 1992. The cave-audio visual experience virtual environment. Comm. ACM 35, 6, 65--72.

[11]

Dahley, A., Wisneski, C., and Ishii, H. 1998. Water lamp and pinwheels: Ambient projection of digital information into architectural space. InProceedings of the Conference on Human Factors in Computing Systems. Los Angeles, CA. 269--270.

[12]

Davis, J. and Chen, X. 2002. LumiPoint: Multi-user location-based interaction on large tiled displays. Elsevier Science 23, 5.

[13]

Ehn, P. and Kyng, M. Cardboard computers: Mocking-it-up or hands-on the future. In Design at Work: Cooperative Design of Computer Systems. J. Greenbaum and M. Kyng, Eds. Lawrence Erlbaum Ass., Hillsdale, CA, 169--197.

[14]

Foley, J. D., Wallace, V. L., and Chan, P. 1984. The human factors of computer graphics interaction techniques. IEEE Comput. Graph. Appl. 4, 11, 13--48.

[15]

Gaver, W. and Martin, H. 2000. Alternatives: Exploring information appliances through conceptual design proposals. In Proceedings of the Conference on Human Factors in Computing Systems. The Hague, Netherlands. 209--216.

[16]

Gaver, W., Beaver, J., and Benford S. 2003. Ambiguity as a resource for design. In Proceedings of the Conference on Human Factors in Computing Systems. For Lauderdale, FL. 233--240.

[17]

Gaver, W. 1992. The affordances of media spaces. In Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work. Toronto, Canada. 1, 7--24.

[18]

Gaver, W. and Dunne, A. 1999. Projected realities: Projected design for cultural effect. In Proceedings of the Conference on Human Factors in Computing Systems. Pittsburgh, PA. 600--607.

[19]

Ghali, A., Boumi, S., Benford, S., Green, J., and Pridmore, T. 2003. Visually tracked flashlights as interaction devices. In Proceedings of Interact. Zurich, Switzerland. 487--494.

[20]

Gibson, J. J. 1977. The Theory of Affordances. In Perceiving, Acting and Knowing. R. E. Sahw & J. Bransford, Eds. Lawrence Erlbaum Ass. Hillsdale, CA. 67--82.

[21]

Green, J., Schnädelbach, H., Koleva, B., Benford S., Pridmore, T., Medina, K., Harris, E., and Smith, H. 2002. Camping in the digital wilderness: Tents and flashlights as interfaces to virtual worlds. In Proceedings of the Conference on Human Factors in Computing Systems Extended Abstracts. Minneapolis, MN. 780--781.

[22]

Griffith, N. and Fernstrom, M. 1998. Litefoot---a floor space for recording dance and controlling media. In Proceedings of the International Computer Music Conference. Ann Arbor, MI. 475--481.

[23]

Hughes, J. A., Randall, D., and Shapiro, D. 1992. Faltering from ethnography to design. In Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work. Toronto, Canada. 115--122.

[24]

Ishii, H. and Ullmer, B. 1997. Tangible bits: Towards seamless interfaces between people, bits and atoms. In Proceedings of the Conference on Human Factors in Computing Systems. Atlanta, GA. 234--241.

[25]

Ishii, H., Orbanes, J., Chun, B., and Paradiso, J. 1999. PingPongPlus: Design of an athletic-tangible interface for computer-supported cooperative play. In Proceedings of the Conference on Human Factors in Computing Systems. Pittsburgh, PA. 394--401.

[26]

Jacob, R. J. K., Sibert, L. E., Mcfarlane, D. C., and Mullen, M. P. 1994. Integrality and separability of input devices. ACM Trans. Comput.-Hum. Interact. 1, 1, 3--26.

[27]

John, B. and Kieras, D. 1996. The GOMS family of user interface analysis techniques: Compare and contrast. ACM Trans. Comput.-Hum. Interact. 3, 4, 320--351.

[28]

Klemmer, S. R., Thomsen, M., Phelps-Goodman, E., Lee, R., and Landay, J. A. 2002. Where do web sites come from?: Capturing and interacting with design theory. In Proceedings of the Conference on Human Factors in Computing Systems. Minneapolis, MN. 1--8.

[29]

Koleva, B., Schnädelbach, H, Flintham, Fraser M., Izadi S., Chandler P., Foster M., Benford S., Greenhalgh C. and Rodden T. 2002. The Augurscope: A mixed reality interface for outdoors. In Proceedings of the Conference on Human Factors in Computing Systems. Minneapolis, MN. 9--16.

[30]

Mackenzie, I. S., Soukoreff, R. W., and Pal, C. 1997. A two-ball mouse affords three degrees of freedom. In Proceedings of the Conference on Human Factors in Computing Systems Conference Companion. New York, NY. 303--304.

[31]

Mackinlay, J. D., Card, S. K., and Robertson, G. G. 1990. A semantic analysis of the design space of input devices. Hum.-Comput. Interact. 5, 23, 145--190.

[32]

Mueller, F., Agamanolis, S., and Picard, R. 2003. Exertion interfaces: Sports over a distance for social bonding and fun, In Proceedings of the Conference on Human Factors in Computing Systems. For Lauderdale. FL. 651--568.

[33]

Myers, B., Bhatnager, R., Nichols, J., Peck, C., King D., Miller, R., and Long, C. 2002. Interacting at a distance: Measuring the performance of laser pointers and other devices. In Proceedings of the Conference on Human Factors in Computing Systems. Minneapolis, MN. 33--40.

[34]

Norman, D. A. 1988. The Psychology of Everyday Things. Basic Books, New York, NY.

[35]

Norman, D. A. 1999. Affordances, conventions and design. Interact. 6, 3, 38--43.

[36]

Olsen, D. and Nielsen, T. 2001. Laser pointer interaction. In Proceedings of the Conference on Human Factors in Computing Systems. Seattle, WA. 17--22.

[37]

Paulos, E. and Canny, J. 1998. PRop: Personal roving presence. In Proceedings of the Conference on Human Factors in Computing Systems. Los Angeles, CA. 296--303.

[38]

Poupyrev, I., Billinghurst, M., Weghorst, S., and Ichikawa, T. 1996. Go go interaction technique: Non-linear mapping for direct manipulation in VR. In Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology. Banff, Canada. 79--80.

[39]

Rekimoto, J. and Sciammarella, E. 2000. ToolStone: Effective use of the physical manipulation vocabularies of input devices. In Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology. San Diego, CA. 109--117.

[40]

Schmidt, A., Strohbach, M., Van Leerhoven, K., Friday, A., and Gellersen, H. 2002. Context acquisition based on load sensing. In Proceedings of the International Conference on Ubiquitous Computing. Goteborg, Sweden. 333--350.

[41]

Sudnow, D. 1978. Ways of the Hand: The Organization of Improvised Conduct. Harvard University Press, Cambridge, MA.

[42]

Ullmer, B., Ishii, H., and Glas, D. 1998. mediaBlocks: Physical containers, transports, and controls for online media. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. Orlando, FL. 379--386.

[43]

Underkoffler, J. and Ishii, H. 1999. Urp: A luminous-tangible workbench for urban planning and design. In Proceedings of the Conference on Human Factors in Computing Systems. Pittsburgh, PA. 386--393.

Cited By

Shapiro KQuijada Leyton PStemple LUberti D(2024)Interacting from Afar: A Study of the Relationship Between Usability and Presence in Object Selection at a DistanceProceedings of 21st ACM SIGGRAPH Conference on Visual Media Production10.1145/3697294.3697305(1-9)Online publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1145/3697294.3697305
Mueller FSemertzidis NAndres JMarshall JBenford SLi XMatjeka LMehta Y(2023)Toward Understanding the Design of Intertwined Human–Computer IntegrationsACM Transactions on Computer-Human Interaction10.1145/359076630:5(1-45)Online publication date: 23-Sep-2023
https://dl.acm.org/doi/10.1145/3590766
Turmo Vidal LMárquez Segura EWaern A(2023)Intercorporeal Biofeedback for Movement LearningACM Transactions on Computer-Human Interaction10.1145/3582428Online publication date: 30-Jan-2023
https://doi.org/10.1145/3582428
Show More Cited By

Index Terms

Expected, sensed, and desired: A framework for designing sensing-based interaction
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI theory, concepts and models
    2. Interaction paradigms
  2. Interaction design
    1. Interaction design process and methods
      1. User centered design
    2. Interaction design theory, concepts and paradigms

Recommendations

VROOM: Virtual Robot Overlay for Online Meetings
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

Telepresence robots allow users to freely explore a remote space and provide a physical embodiment in that space. However, they lack a compelling representation of the remote user in the local space. We present VROOM (Virtual Robot Overlay for Online ...
Garden: A Mixed Reality Experience Combining Virtual Reality and 3D Reconstruction
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Garden is a Mixed Reality (MR) experience that combines both Virtual Reality (VR) and Augmented Reality (AR), and lets players transform their environment into a virtual garden they can play in. This is done by doing both stereoscopic rendering and 3D ...
Mini-Me: An Adaptive Avatar for Mixed Reality Remote Collaboration
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

We present Mini-Me, an adaptive avatar for enhancing Mixed Reality (MR) remote collaboration between a local Augmented Reality (AR) user and a remote Virtual Reality (VR) user. The Mini-Me avatar represents the VR user's gaze direction and body gestures ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction

ACM Transactions on Computer-Human Interaction Volume 12, Issue 1

March 2005

146 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/1057237

Issue’s Table of Contents

Copyright © 2005 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 March 2005

Published in TOCHI Volume 12, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

142
Total Citations
View Citations
4,190
Total Downloads

Downloads (Last 12 months)117
Downloads (Last 6 weeks)16

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Shapiro KQuijada Leyton PStemple LUberti D(2024)Interacting from Afar: A Study of the Relationship Between Usability and Presence in Object Selection at a DistanceProceedings of 21st ACM SIGGRAPH Conference on Visual Media Production10.1145/3697294.3697305(1-9)Online publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1145/3697294.3697305
Mueller FSemertzidis NAndres JMarshall JBenford SLi XMatjeka LMehta Y(2023)Toward Understanding the Design of Intertwined Human–Computer IntegrationsACM Transactions on Computer-Human Interaction10.1145/359076630:5(1-45)Online publication date: 23-Sep-2023
https://dl.acm.org/doi/10.1145/3590766
Turmo Vidal LMárquez Segura EWaern A(2023)Intercorporeal Biofeedback for Movement LearningACM Transactions on Computer-Human Interaction10.1145/3582428Online publication date: 30-Jan-2023
https://doi.org/10.1145/3582428
Nordmoen CMcPherson A(2023)What the Sensor Knows: More-Than-Human Knowledge Co-Production in Wood CarvingProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3596075(779-789)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3596075
Mueller Fvon Kaenel JDuckworth JAndres J(2023)Taking inspiration from becoming “one with a bike” to design human-computer integrationHuman–Computer Interaction10.1080/07370024.2023.2269148(1-26)Online publication date: 3-Nov-2023
https://doi.org/10.1080/07370024.2023.2269148
Ambe ASoro AJohnson DBrereton M(2022)From Collaborative Habituation to Everyday Togetherness: A Long-Term Study of Use of the Messaging KettleACM Transactions on Computer-Human Interaction10.1145/347097329:1(1-47)Online publication date: 7-Jan-2022
https://dl.acm.org/doi/10.1145/3470973
Cardoso JRibeiro J(2021)Tangible VR Book: Exploring the Design Space of Marker-Based Tangible Interfaces for Virtual RealityApplied Sciences10.3390/app1104136711:4(1367)Online publication date: 3-Feb-2021
https://doi.org/10.3390/app11041367
Kong AAhuja KGoel MHarrison C(2021)EyeMU Interactions: Gaze + IMU Gestures on Mobile DevicesProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3479938(577-585)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3462244.3479938
Kim KPark MLim Y(2021)Guiding Preferred Driving Style Using Voice in Autonomous Vehicles: An On-Road Wizard-of-Oz StudyProceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462056(352-364)Online publication date: 28-Jun-2021
https://dl.acm.org/doi/10.1145/3461778.3462056
Genç ZLenzini GSgandurra D(2021)Cut-and-Mouse and Ghost ControlDigital Threats: Research and Practice10.1145/34312862:1(1-23)Online publication date: 11-Feb-2021
https://dl.acm.org/doi/10.1145/3431286
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents