Article

Free access

Speech and gestures for graphic image manipulation

Author:

A. G. HauptmannAuthors Info & Claims

CHI '89: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 241 - 245

https://doi.org/10.1145/67449.67496

Published: 01 March 1989 Publication History

Abstract

An experiment was conducted with people using gestures and speech to manipulate graphic images on a computer screen. A human was substituted for the recognition devices. The analysis showed that people strongly prefer to use both gestures and speech for the graphics manipulation and that they intuitively use multiple hands and multiple fingers in all three dimensions. There was surprising uniformity and simplicity in the gestures and speech. The analysis of these results provides strong encouragement for future development of integrated multi-modal interaction systems.

References

[1]

Birdwhistell, R.L. Kinesics and context; essays on body motion communication. University of Pennsylvania Press, Philadelphia, 1970.]]

[2]

Dragon Systems, Inc. VoiceScribe-lO00: Speech Recognition System. Newton, MA, 1986.]]

[3]

Ford, W.R., Weeks, G.D. and Chapanis, A. "The effect of serf-imposed brevity on the structure of dyadic communication". Journal of Psychology 104 (1980), 87 - 103.]]

[4]

Friedman, L.A. On the other Hand. Academic Press, New York, 1977.]]

[5]

Fromkin, V. and Rodman, R. An Introduction to Language. Rhinehart and Winston, New York, 1978.]]

Digital Library

[6]

VPL Research, Inc. Dataglove Model 2 System. Redwood City, CA, 1987.]]

[7]

Greene, R. "The drawing prism: A Versatile Graphic input Device". ACM SIGGRAPH '85 19, 3 (1985), 103 - 110.]]

Digital Library

[8]

Hauptmann, A.G., McAvinney, P. and Shepard, S. Gesture Analysis for Graphic Manipulation. Department of Computer Science CMU-CS-88-198, Carnegie-Mellon University, November, 1988.]]

[9]

Hewes, G. "The current Status of the Gestural Theory of Language". Annals of the New York Academy of Science 280 (~976).]]

[10]

Johnstone, E. The Rolky: A Poly-Touch Controller for Electronic Music. ICMC Proceedings, 1985, pp. 291 - 295.]]

[11]

Marlin, J. Design of Man-Computer Dialogues. Prentice-Hall, Englewood Cliffs, 1973.]]

Digital Library

[12]

Lee, S.K., Buxton, W. and Smith, K.C. A Multi-Touch Three Dimensional Touch-Sensitive Tablet. CHI-85 Conference Proceedings: Human Factors in Computing Systems, ACM, San Francisco, CA, April, 1985, pp. 21 - 25.]]

Digital Library

[13]

Myers, J.L. Fundamentals of Experimental Design. Allyn and Bacon, Boston, MA, 1972.]]

[14]

Paget, R. Human Speech. Harcourt, Brace, Jovanovich, New York, 1930.]]

[15]

McAvinney, P. The Sensor Frame - A Gesture-Based Device for the Manipulation of Graphic Objects. Carnegie- Mellon University, 1986.]]

[16]

Shneiderman, B. Software Psychology. Winthrop, Cambridge, 1980.]]

Cited By

Masson DMalacria SCasiez GVogel D(2024)DirectGPT: A Direct Manipulation Interface to Interact with Large Language ModelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642462(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642462
Mich OSchiavo GFerron MMana N(2022)Framing the Design Space of Multimodal Mid-Air Gesture and Speech-Based Interaction With Mobile Devices for Older PeopleResearch Anthology on Supporting Healthy Aging in a Digital Society10.4018/978-1-6684-5295-0.ch028(496-519)Online publication date: 4-Feb-2022
https://doi.org/10.4018/978-1-6684-5295-0.ch028
Aziz FCreed CFrutos-Pascual MWilliams I(2021)Inclusive Voice Interaction Techniques for Creative Object PositioningProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3479937(461-469)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3462244.3479937
Show More Cited By

Index Terms

Speech and gestures for graphic image manipulation
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Speech recognition
  2. Computer graphics
    1. Graphics systems and interfaces
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Graphics input devices

Recommendations

Speech and gestures for graphic image manipulation

An experiment was conducted with people using gestures and speech to manipulate graphic images on a computer screen. A human was substituted for the recognition devices. The analysis showed that people strongly prefer to use both gestures and speech for ...
A graphic object manipulation system using hand gestures
CHI '92: Posters and Short Talks of the 1992 SIGCHI Conference on Human Factors in Computing Systems

People use gestures to communitcate with other people without difficutlty. As a method of computer/human interaction, gestures are more intuitive and more direct than existing input devices, such as keyboards, mice, etc., and futhermore, are more ...
Eye-based head gestures
ETRA '12: Proceedings of the Symposium on Eye Tracking Research and Applications

A novel method for video-based head gesture recognition using eye information by an eye tracker has been proposed. The method uses a combination of gaze and eye movement to infer head gestures. Compared to other gesture-based methods a major advantage ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '89: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

March 1989

397 pages

ISBN:0897913019

DOI:10.1145/67449

Editors:
K. Bice
Texas Instruments
,
C. Lewis
Univ. of Colorado, Boulder, CO

ACM SIGCHI Bulletin Volume 20, Issue SI
March 1989
374 pages
ISSN:0736-6906
DOI:10.1145/67450
Editors:
K. Bice
Texas Instruments
,
C. Lewis
Univ. of Colorado, Boulder, CO
Issue’s Table of Contents

Copyright © 1989 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 March 1989

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

CHI '89 Paper Acceptance Rate 54 of 199 submissions, 27%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

108
Total Citations
View Citations
1,467
Total Downloads

Downloads (Last 12 months)119
Downloads (Last 6 weeks)19

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Masson DMalacria SCasiez GVogel D(2024)DirectGPT: A Direct Manipulation Interface to Interact with Large Language ModelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642462(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642462
Mich OSchiavo GFerron MMana N(2022)Framing the Design Space of Multimodal Mid-Air Gesture and Speech-Based Interaction With Mobile Devices for Older PeopleResearch Anthology on Supporting Healthy Aging in a Digital Society10.4018/978-1-6684-5295-0.ch028(496-519)Online publication date: 4-Feb-2022
https://doi.org/10.4018/978-1-6684-5295-0.ch028
Aziz FCreed CFrutos-Pascual MWilliams I(2021)Inclusive Voice Interaction Techniques for Creative Object PositioningProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3479937(461-469)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3462244.3479937
Mich OSchiavo GFerron MMana N(2020)Framing the Design Space of Multimodal Mid-Air Gesture and Speech-Based Interaction With Mobile Devices for Older PeopleInternational Journal of Mobile Human Computer Interaction10.4018/IJMHCI.202001010212:1(22-41)Online publication date: 1-Jan-2020
https://doi.org/10.4018/IJMHCI.2020010102
Johnston M(2019)Multimodal integration for interactive conversational systemsThe Handbook of Multimodal-Multisensor Interfaces10.1145/3233795.3233798(21-76)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1145/3233795.3233798
Ito ANakada K(2019)UI Design based on Traditional Japanese Gesture2019 10th IEEE International Conference on Cognitive Infocommunications (CogInfoCom)10.1109/CogInfoCom47531.2019.9089950(85-90)Online publication date: Oct-2019
https://doi.org/10.1109/CogInfoCom47531.2019.9089950
Gygli MFerrari V(2019)Fast Object Class Labelling via Speech2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR.2019.00551(5360-5368)Online publication date: Jun-2019
https://doi.org/10.1109/CVPR.2019.00551
Hansberger JPeng CBlakely VMeacham SCao LDiliberti N(2019)A Multimodal Interface for Virtual Information EnvironmentsVirtual, Augmented and Mixed Reality. Multimodal Interaction10.1007/978-3-030-21607-8_5(59-70)Online publication date: 8-Jun-2019
https://doi.org/10.1007/978-3-030-21607-8_5
Srinivasan AStasko J(2018)Orko: Facilitating Multimodal Interaction for Visual Exploration and Analysis of NetworksIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2017.274521924:1(511-521)Online publication date: Jan-2018
https://doi.org/10.1109/TVCG.2017.2745219
Johnston M(2016)Extensible Multimodal Annotation for Intelligent Interactive SystemsMultimodal Interaction with W3C Standards10.1007/978-3-319-42816-1_3(37-64)Online publication date: 18-Nov-2016
https://doi.org/10.1007/978-3-319-42816-1_3
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents