Nothing Special   »   [go: up one dir, main page]

skip to main content
article

Emerging frameworks for tangible user interfaces

Published: 01 July 2000 Publication History

Abstract

We present steps toward a conceptual framework for tangible user interfaces. We introduce the MCRpd interaction model for tangible interfaces, which relates the role of physical and digital representations, physical control, and underlying digital models. This model serves as a foundation for identifying and discussing several key characteristics of tangible user interfaces. We identify a number of systems exhibiting these characteristics, and situate these within 12 application domains. Finally, we discuss tangible interfaces in the context of related research themes, both within and outside of the human-computer interaction domain.

References

[1]
1. G. Fitzmaurice, H. Ishii, and W. Buxton, "Bricks: Laying the Foundations for Graspable User Interfaces," Proceedings of CHI'95 (1995), pp. 442-449.]]
[2]
2. G. Fitzmaurice, Graspable User Interfaces, Ph.D. thesis, University of Toronto, Toronto (1996).]]
[3]
3. H. Ishii and B. Ullmer, "Tangible Bits: Towards Seamless Interfaces Between People, Bits, and Atoms," Proceedings of CHI'97 (1997), pp. 234-241.]]
[4]
4. J. Underkoffler and H. Ishii, "Urp: A Luminous-Tangible Workbench for Urban Planning and Design," Proceedings of CHI'99 (1999), pp. 386-393.]]
[5]
5. J. Underkoffler, B. Ullmer, and H. Ishii, "Emancipated Pixels: Real-World Graphics in the Luminous Room," Computer Graphics Proceedings (SIGGRAPH 99) (1999), pp. 385-392.]]
[6]
6. S. Burbeck, Applications Programming in Smalltalk-80: How to Use Model-View-Controller (1987), see http:// st-www.cs.uiue.edu/users/smarch/st-does/mvc.html.]]
[7]
7. J. Coutaz, "PAC, an Object Oriented Model for Dialog Design," Proceedings of Interact'87 (1987), pp. 431-436.]]
[8]
8. The "tangibles" term was used in this context ca. 1994 at Interval Research Corp., associated with the development of the LogJam video logging and ToonTown audio conferencing systems.]]
[9]
9. J. Cohen, M. Withgott, and P. Piernot, "LogJam: A Tangible Multi-Person Interface for Video Logging," Proceedings of CHI'99 (1999).]]
[10]
10. A. Singer, D. Hindus, L. Stifelman. and S. White, "Tangible Progress: Less Is More in Somewire Audio Spaces," Proceedings of CHI'99 (1999).]]
[11]
11. K. MacLean, S. Snibbe, and G. Levin, "Tagged Handles: Merging Discrete and Continuous Manual Control," Proceedings of CHI'00 (2000), pp. 225-232.]]
[12]
12. D. Reznick, E. Moshkovich, and J. Canny, "Building a Universal Part Manipulator," Distributed Manipulation, Bohringer and Choset, Editors, Kluwer Academic Press, Dordrecht, the Netherlands (1999).]]
[13]
13. B. Ullmer, H. Ishii, and D. Glas, "mediaBlocks: Physical Containers, Transports, and Controls for Online Media," Computer Graphics Proceedings (SIGGRAPH 98) (1998), pp. 379-386.]]
[14]
14. B. Ullmer and H. Ishii, "mediaBlocks: Tangible Interfaces for Online Media," CHI'99 Extended Abstracts (video demonstration) (1999), pp. 31-32.]]
[15]
15. B. Ullmer and H. Ishii, "The metaDESK: Models and Prototypes for Tangible User Interfaces," Proceedings of UIST'97 (1997), pp. 223-232.]]
[16]
16. S. Houde and G. Salomon, "Working Towards Rich and Flexible File Representations," Proceedings of INTERCHI'93, Adjunct Proceedings (1993), pp. 9-10.]]
[17]
17. J. Johnson, T. Roberts, W. Verplank, et al., "The Xerox Star: A Retrospective," Computer22, No. 9, 11-29 (September 1989).]]
[18]
18. M. Familant and M. Detweiler, "Iconic Reference: Evolving Perspectives and an Organising Framework," International Journal of Man-Machine Studies39, 705-728 (1993).]]
[19]
19. M. Gorbet, Beyond Input Devices: A New Conceptual Framework for the Design of Physical-Digital Objects, M.S. thesis, MIT Media Laboratory, Cambridge, MA (1998).]]
[20]
20. S. McCloud, Understanding Comics: The Invisible Art, Rutgers University Press, London (1993).]]
[21]
21. G. Crampton-Smith, "The Hand That Rocks the Cradle," I.D., 60-65 (May/June 1995).]]
[22]
22. L. Holmquist. J. Redström, and P. Ljungstrand, "Token-Based Access to Digital Information," Proceedings of HUC'99 (1999), pp. 234-245.]]
[23]
23. B. Ullmer, Models and Mechanisms for Tangible User Interfaces, M.S. thesis, MIT Media Laboratory, Cambridge, MA (June 1997).]]
[24]
24. E. Arias, H. Eden, and G. Fisher, "Enhancing Communication, Facilitating Shared Understanding, and Creating Better Artifacts by Integrating Physical and Computational Media for Design," Proceedings of DIS'97 (1997), pp. 1-12.]]
[25]
25. K. Schäfer, V. Brauer, and W. Bruns, "A New Approach to Human-Computer Interaction--Synchronous Modelling in Real and Virtual Spaces," Proceedings of DIS'97 (1997), pp. 335-344.]]
[26]
26. J. Underkoffler and H. Ishii, "Illuminating Light: An Optical Design Tool with a Luminous-Tangible Interface," Proceedings of CHI'98 (1998), pp. 542-549.]]
[27]
27. H. Shwe, Smarter Play for Smart Toys: The Benefits of Technology-Enhanced Play (1999), at http://www.zowie.com/ about/smart.html.]]
[28]
28. M. Fjeld, M. Bichsel, and M. Rauterberg, "BUILD-IT: An Intuitive Design Tool Based on Direct Object Manipulation," Gesture and Sign Language in Human-Computer Interaction, Lecture Notes in Artificial Intelligence, Vol. 1371, Wachsmut and Fröhlich, Editors, Springer-Verlag, Berlin (1998), pp. 297-308.]]
[29]
29. I. Siio, "InfoBinder: A Pointing Device for a Virtual Desktop System," Proceedings' of the IHCI'95 (1995).]]
[30]
30. K. Hinckley, R. Pausch, J. Goble, and N. Kassel, "Passive Real-World Interface Props for Neurosurgical Visualization," Proceedings of CHI'94 (1994), pp. 452-458.]]
[31]
31. D. Small, Rethinking the Book, Ph.D. thesis, MIT Media Laboratory, Cambridge, MA (1999).]]
[32]
32. P. Wellner, "Interacting with Paper on the DigitalDesk," Communications of the ACM36, No. 7, 86-96 (July 1993).]]
[33]
33. R. Aish, "Three-Dimensional Input for CAAD Systems," Computer-Aided Design11, No. 2, 66-70 (March 1979).]]
[34]
34. R. Aish and P. Noakes, "Architecture Without Numbers-- CAAD Based on a 3D Modelling System," Computer-Aided Design16, No. 6, 321-328 (November 1984).]]
[35]
35. J. Frazer, "Three-Dimensional Data Input Devices," Computers/Graphics in the Building Process, Washington (1982).]]
[36]
36. J. Frazer, An Evolutionary Architecture, Architectural Association, London (1994).]]
[37]
37. G. Anagnostou, D. Dewey, and A. Patera, "Geometry-Defining Processors for Engineering Design and Analysis," The Visual Computer5, 304-315 (1989).]]
[38]
38. M. Gorbet, M. Orth, and H. Ishii, "Triangles: Tangible Interface for Manipulation and Exploration of Digital Information Topography," Proceedings of CHI'98 (1998), pp. 49-56.]]
[39]
39. K. Kramer, Moveable Objects, Mobile Code, M.S. thesis, MIT Media Laboratory, Cambridge, MA (1998).]]
[40]
40. M. Resnick, F. Berg, et al., "Digital Manipulatives: New Toys to Think With," Proceedings of CHI'98 (1998), 281-287.]]
[41]
41. K. Kramer and N. Minar, Stackables: Manipulable Distributed Displays (1997), see http://el.www.media.mit.edu/ projects/stackables/.]]
[42]
42. K. Heaton, R. Poor, and A. Wheeler, "Nami," Conference Abstracts and Applications of SIGGRAPH 99, Emerging Technologies (1999), p. 214.]]
[43]
43. D. Anderson, J. Frankel, J. Marks, et al., "Building Virtual Structures with Physical Blocks" (Demo Description), Proceedings of UIST'99 (1999), pp. 71-72.]]
[44]
44. T. McNerney, Tangible Programming Bricks: An Approach to Making Programming Accessible to Everyone, M.S. thesis, MIT Media Laboratory, Cambridge, MA (2000).]]
[45]
45. R. Perlman, "Using Computer Technology to Provide a Creative Learning Environment for Preschool Children," MIT Logo Memo #24, Cambridge, MA (1976).]]
[46]
46. H. Suzuki and H. Kato, "AlgoBlock: A Tangible Programming Language, a Tool for Collaborative Learning," Proceedings of the Fourth European Logo Conference, Athens, Greece (August 1993), pp. 297-303.]]
[47]
47. R. Abrams, Adventures in Tangible Computing: The Work of Interaction Designer Durrell Bishop, in Context, M.A. thesis, Royal College of Art, London (1999).]]
[48]
48. L. Nelson, S. Ichimura, E. Pedersen, and L. Adams, "Palette: A Paper Interface for Giving Presentations," Proceedings of CHI'99 (1999), pp. 354-361.]]
[49]
49. H. Ishii, R. Fletcher, et al., "musicBottles," Abstracts and Applications of SIGGRAPH 99, Emerging Technologies (1999), p. 174.]]
[50]
50. N. Jeremijenko, "Voice Boxes: Memesis," Ars Electronica'96 Festival Catalog, Springer, New York (1996), p. 403.]]
[51]
51. R. Borovoy, Genuine Object-Oriented Programming, M.S. thesis, MIT Media Laboratory, Cambridge, MA (1995).]]
[52]
52. M. Umaschi, "Soft Toys with Computer Hearts: Building Personal Storytelling Environments," CHI'97 Extended Abstracts (1997), pp. 20-21.]]
[53]
53. J. Glos and J. Cassell, "Rosebud: Technological Toys for Storytelling," CHI'97 Extended Abstracts (1997), pp. 359-360.]]
[54]
54. N. Streitz, J. Geißler, et al., "i-LAND: An Interactive Landscape for Creativity and Innovation," Proceedings" of CHI'99 (1999), pp. 120-127.]]
[55]
55. M. Resnick, "Behavior Construction Kits," Communications of the ACM36, No. 7, 64-71 (July 1993).]]
[56]
56. P. Frei, V. Su, B. Mikhak, and H. Ishii, "curlybot: Designing a New Class of Computational Toys," Proceedings of CHI'00 (2000), pp. 129-136.]]
[57]
57. A. Dunne and F. Raby, Fields and Thresholds, Presentation at the Doors of Perception 2 (November 1994), see http:// www.mediamatic.nl/doors/Doors2/Doors2Fry.html.]]
[58]
58. M. Weiser and J. S. Brown, The Coming Age of Calm Technology (October 5, 1996), see http://www.ubiq.com/hypertext/ weiser/acmfuture2endnote.htm.]]
[59]
59. C. Wisneski, H. Ishii, A. Dahley, M. Gorbet, S. Brave, B. Ullmer, and P. Yarin "Ambient Displays: Turning Architectual Space into an Interface Between People and Digital Information," Proceedings of CoBuild '98 (1998), pp. 22-32.]]
[60]
60. E. Pedersen and T. Sokoler, "AROMA: Abstract Representation of Presence Supporting Mutual Awareness," Proceedings of CHI'97 (1997), pp. 51-58.]]
[61]
61. A. Dahley, C. Wisneski, and H. Ishii, "Water Lamp and Pinwheels: Ambient Projection of Digital Information into Architectural Space," Proceedings of CHI'98 Companion (1998), pp. 269-270.]]
[62]
62. H. Kuzuoka and S. Greenberg, "Mediating Awareness and Communication Through Digital But Physical Surrogates," Proceedings of CHI'99 Companion (1999), pp. 11-12.]]
[63]
63. C. Wisneski, The Design of Personal Ambient Displays, M.S. thesis, MIT Media Laboratory, Cambridge, MA (1999).]]
[64]
64. J. Heiner, S. Hudson, and K. Tanaka, "The Information Percolator: Ambient Information Display in a Decorative Object," Proceedings of UIST'99 (1999), pp. 141-148.]]
[65]
65. S. Brave, H. Ishii, and A. Dahley, "Tangible Interfaces for Remote Collaboration and Communication," Proceedings of CSCW '98 (1998), pp. 169-178.]]
[66]
66. W. Mackay and D. Pagani, "Video Mosaic: Laying Out Time in a Physical Space," Proceedings of Multimedia'94 (1994), pp. 165-172.]]
[67]
67. T. Arai, K. Machii, and S. Kuzunuki, "Retrieving Electronic Documents with Real-World Objects on the InteractiveDESK," Proceedings of UIST'95 (1995), pp. 37-38.]]
[68]
68. L. Stifelman, "Augmenting Real-World Obiects: A Paper-Based Audio Notebook," Proceedings of CHI'96 Companion (1996), pp. 199-200.]]
[69]
69. H. Ishii, C. Wisneski, J. Orbanes, B. Chun, and J. Paradiso, "PingPongPlus: Design of an Athletic-Tangible Interface for Computer-Supported Cooperative Play," Proceedings of CHI '99 (1999), pp. 394-401.]]
[70]
70. P. Yarin and H. Ishii, "TouchCounters: Designing Interactive Electronic Labels for Physical Containers," Proceedings of CHI'99 (1999), pp. 362-368.]]
[71]
71. R. Want, K. Fishkin, A. Gujar, and B. Harrison, "Bridging Physical and Virtual Worlds with Electronic Tags," Proceedings of CHI'99 (1999), pp. 370-377.]]
[72]
72. J. Rekimoto and M. Saitoh, "Augmented Surfaces: A Spatially Continuous Work Space for Hybrid Computing Environments," Proceedings of CHI'99 (1999), pp. 378-385.]]
[73]
73. B. Ullmer, Cognitive Roles of Physical Artifacts in Non-Athletic Games, unpublished manuscript (2000).]]
[74]
74. M. Csikszentmihalyi and E. Rochberg-Halton, The Meaning of Things: Domestic Symbols and the self. Cambridge University Press, Cambridge, UK (1981).]]
[75]
75. J. González, "Autotopographics," Prosthetic Territories: Politics and Hypertechnologies, Westview Press, Boulder, CO. pp. 133-150.]]
[76]
76. J. Zhang, "The Nature of External Representations in Problem Solving," Cognitive Science21, No. 2, 179-217 (1997).]]
[77]
77. J. Zhang and D. Norman, "Representations in Distributed Cognitive Tasks," Cognitive Science18, 87-122 (1994).]]
[78]
78. J. Gibson, The Ecological Approach to Visual Perception. Erlbaum Associates. New York (1979).]]
[79]
79. D. Norman, Things That Make Us Smart, Addison-Wesley Publishing Co., Reading, MA (1993).]]
[80]
80. E. Hutchins, Cognition in the Wild, MIT Press. Cambridge, MA (1995).]]
[81]
81. M. Scaife and Y. Rogers, "External Cognition: How Do Graphical Representations Work?" International Journal of Human-Computer Studies45, No. 2, 185-213 (1996).]]
[82]
82. N. Eilan, R. McCarthy, and B. Brewer, Spatial Representation, Blackwell. Oxford, UK (1993).]]
[83]
83. D. Kirsh, "The Intelligent Use of Space," Artificial Intelligence (1995).]]
[84]
84. Y. Guiard, "Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model," The Journal of Motor Behavior19, No. 4, 486-517 (1987).]]
[85]
85. K. Hinckley et al., "Two-Handed Virtual Manipulation," ACM TOCHI'98 (1998).]]
[86]
86. M. Krampen, "Objects," Encyclopedic Dictionary of Semiotics, Vol 1, T. Sebcok, Editor, Mouton de Gruyter, New York (1986), pp. 635-639.]]
[87]
87. J. McCarthy, An Introduction to Theoretical Kinematics, MIT Press, Cambridge, MA (1990).]]
[88]
88. Semantic Visions in Design, S. Vihma, Editor, UIAH, Helsinki (1990).]]
[89]
89. B. Shneiderman, "Direct Manipulation: A Step Beyond Programming Languages," Computer16, 57-69 (1983).]]
[90]
90. D. Frohlich, "Direct Manipulation and Other Lessons," Handbook of Human-Computer Interaction, 2e, Ch. 21, Elsevier, Amsterdam (1997).]]
[91]
91. D. Gentner and J. Nielsen, "'The Anti-Mac Interface," Communications of the ACM39, No. 8, 70-82 (August 1996).]]
[92]
92. R. Baeza-Yates, Visual Programming (1998), formerly at http:// www-lsi.upc.es/~rbaeza/sem/vp/vp.html. (Visited August 19, 1998.)]]
[93]
93. J. Larkin and H. Simon, "Why a Diagram Is (Sometimes) Worth Ten Thousand Words," Cognitive Science11, 65-99 (1987).]]
[94]
94. M. Petre, "Why Looking Isn't Always Seeing: Readership Skills and Graphical Programming," Communications of the ACM38, 33-44 (June 1995).]]
[95]
95. S. Feiner, B. MacIntyre, and D. Seligmann, "Knowledge-Based Augmented Reality," Communications of the ACM36, No. 7, 52-62 (July 1993).]]
[96]
96. R. Azuma, "A Survey of Augmented Reality," Presence6, No. 4, 355-385 (August 1997).]]
[97]
97. W. Mackay, "Augmented Reality: Linking Real and Virtual Worlds," (published keynote address), Proceedings of ACM AVI'98 (1998).]]
[98]
98. P. Milgram and F. Kishino, "A Taxonomy of Mixed Reality Displays," IEICE Transactions on Information Systems, E77-D(12) (December 1994).]]
[99]
99. T. Starner, S. Mann, B. Rhodes. J. Levine, J. Healey, D. Kitsch, R. Picard, and A. Pentland, "Augmented Reality Through Wearable Computing," Presence6, No. 4 (Winter 1997).]]
[100]
100. M. Weiser, "The Computer for the 21st Century," Scientific American265, No. 3, 94-104 (1991).]]
[101]
101. J. R. Cooperstock, K. Tanikoshi, G. Beirne, T. Narine, and W. Buxton, "Evaluation of a Reactive Environment," Proceedings of CHI '95 (1995), pp. 170-177.]]
[102]
102. K. Fishkin, T. Moran, and B. Harrison, "Embodied User Interfaces: Towards Invisible User Interfaces," Proceedings of EHCI'98 (1998).]]
[103]
103. F. Halasz, "Reflections on Notecards: Seven Issues for the Next Generation of Hypermedia Systems," Communications of the ACM31, No. 7, 836-852 (July 1988).]]
[104]
104. F. Halasz," 'Seven Issues' Revisited," keynote address. Hypertext'91 Conference (1991), see http://www.pare.xerox.com/ spl/projects/halasz-keynote/and http://www.csdl.tamu.edu/ ~leggett/halasz.html (video).]]

Cited By

View all
  • (2024)Tangible Affect: A Literature Review of Tangible Interactive Systems Addressing Human Core Affect, Emotions and MoodsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661608(424-440)Online publication date: 1-Jul-2024
  • (2024)Towards Relatable Climate Change Data: Untangling Tensions in Engaging with a HyperobjectProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661606(3029-3045)Online publication date: 1-Jul-2024
  • (2024)Mimosa: Modular Self-folding Hinges Kit for Creating Shape-changing ObjectsProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3633381(1-13)Online publication date: 11-Feb-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IBM Systems Journal
IBM Systems Journal  Volume 39, Issue 3-4
July 2000
517 pages

Publisher

IBM Corp.

United States

Publication History

Published: 01 July 2000

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 14 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Tangible Affect: A Literature Review of Tangible Interactive Systems Addressing Human Core Affect, Emotions and MoodsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661608(424-440)Online publication date: 1-Jul-2024
  • (2024)Towards Relatable Climate Change Data: Untangling Tensions in Engaging with a HyperobjectProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661606(3029-3045)Online publication date: 1-Jul-2024
  • (2024)Mimosa: Modular Self-folding Hinges Kit for Creating Shape-changing ObjectsProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3633381(1-13)Online publication date: 11-Feb-2024
  • (2024)Mediating the Sacred: Configuring a Design Space for Religious and Spiritual Tangible Interactive ArtifactsProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3633353(1-22)Online publication date: 11-Feb-2024
  • (2024)DisplayFab: The State of the Art and a Roadmap in the Personal Fabrication of Free-Form Displays Using Active Materials and Additive Manufacturing.Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642708(1-24)Online publication date: 11-May-2024
  • (2024)A Survey of Designs for Combined 2D+3D Visual RepresentationsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.338851630:6(2888-2902)Online publication date: 22-Apr-2024
  • (2023)Latent Organism: Embodied Co-Creation with AIProceedings of the 15th Conference on Creativity and Cognition10.1145/3591196.3593508(239-242)Online publication date: 19-Jun-2023
  • (2023)Tangible interaction technique with authoring capability for kitchen designMultimedia Tools and Applications10.1007/s11042-023-14376-382:19(30125-30150)Online publication date: 20-Jan-2023
  • (2023)The effect of embodied interaction designs on flow experience: examination in VR gamesVirtual Reality10.1007/s10055-023-00758-327:2(1549-1565)Online publication date: 30-Jan-2023
  • (2023)Local Communities and Their Visitors: An Interaction Design ApproachDesign, User Experience, and Usability10.1007/978-3-031-35699-5_8(94-109)Online publication date: 23-Jul-2023
  • Show More Cited By

View Options

View options

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media