Article

Free access

Communicative facial displays as a new conversational modality

Authors:

Akikazu Takeuchi,

Katashi NagaoAuthors Info & Claims

CHI '93: Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems

Pages 187 - 193

https://doi.org/10.1145/169059.169156

Published: 01 May 1993 Publication History

Abstract

The human face is an independent communication channel that conveys emotional and conversational signals encoded as facial displays. Facial displays can be viewed as communicative signals that help coordinate conversation. We are attempting to introduce facial displays into computer-human interaction as a new modality. This will make the interaction tighter and more efficient while lessening the cognitive load. As the first step, a speech dialogue system was selected to investigate the power of communicative facial displays. We analyzed the conversations between users and the speech dialogue system, to which facial displays had been added. We found that conversation with the system featuring facial displays was more successful than that with a system without facial displays.

References

[1]

Blatmer, M. Multimedia and Multimodal User Interface Design: CHI'92 Tutorial Course Note 4. ACM Press, 1992.

Digital Library

[2]

Chovil, N. Communicative Functions of Facial Displays in Conversation. Ph.D. Thesis, University of Victoria, 1989.

[3]

Darwin, C. The Expression of Emotion in Man and Animals. University of Chicago Press, Chicago, 1965.

[4]

Don, A. and Brennan, S. and Laurel, B. and Shneiderman, B. Anthropomorphism: from Eliza to Terminator 2, In Proc. CHI'92 Human Factors in Computing Systems (Monterey, May 3-7, 1992), ACM Press, pp. 67-70.

Digital Library

[5]

Don, A. and Oren, T. and Laurel, B. GUIDES 3.0, in Proc. CHI'91 Human Factors in Computing Systems (New Orleans, April 27-May 2, 1991), ACM Press, pp. 447-448.

Digital Library

[6]

Ekman, P. and Friesen, W. V. The repertoire of nonverbal behavior - categories, origins, usage, and coding, Semiotica 1 (1969), pp. 49-98.

[7]

Ekman, P. and Friesen, W. V. Facial Action Coding System. Consulting Psychologists Press, Palo Alto, California, 1978.

[8]

Ekman, P. and Friesen, W. V. Unmasking the Face. Consulting Psychologists Press, Inc., Palo Alto, California, 1984.

[9]

Fridlund, A. J. and Gilbert, A. N. Emotions and facial expression, Science, 230 (1985), pp. 607-608.

[10]

Hindus, D. and Brennan, S. Conversational Paradigms in User Interfaces: CHI'92 Tutorial Course Note 11. ACM Press, 1992.

[11]

Itou, K. and Hayamizu, S. and Tanaka, H. Continuous speech recognition by context-dependent phonetic HMM and an efficient algorithm for finding N-best sentence hypotheses, in Proc. ICASSP'92, IEEE Press, pp. 1 21-I 24.

Digital Library

[12]

Nagao, K. A preferential constraint satisfaction technique for natural language analysis, in Proc. ECAI- 92, (1992), pp. 523-527.

Digital Library

[13]

Nagao, K. and Osawa, E. A Logic-Based Approach to Plan Recognition and Belief Revision. Tech. Report. SCSL-TR-92-007, Sony Computer Science Laboratory, Inc., Tokyo, 1992.

[14]

Perret, D. I. et al. Neurones responsive to faces in the temporal cortex: studies of functional organization sensitivity and relation to perception. Human Neurobiology, 3 (1984) 197-208.

[15]

Sherer, K. R. The functions of nonverbal signs in conversation, in The Social and Psychological Contexts of Language, St. Clair, R. N. and Giles, H. ( Eds.), Lawrence Erlbaum, Hillsdale, NJ, 1980, pp. 225-244.

[16]

Takeuchi, A. and Franks, S. A Rapid Face Construction Lab. Tech. Report. SCSL-TR-92-010, Sony Computer Science Laboratory, Inc., Tokyo, 1992.

[17]

Tomita, M. An efficient augmented-context-free parsing algorithm, Computational Linguistics, 13 (1987), pp. 31-46.

Digital Library

[18]

Waters, K. A muscle model for animating threedimensional facial expression, in Computer Graphics 21, 4 (July 1987), 17-24.

Digital Library

Cited By

Leung WWan A(2022)My Little Joy in LifeResearch Anthology on Usage, Identity, and Impact of Social Media on Society and Culture10.4018/978-1-6684-6307-9.ch057(1086-1101)Online publication date: 10-Jun-2022
https://doi.org/10.4018/978-1-6684-6307-9.ch057
Kang BKang BJun SJun S(2022)User Perception of Social Robot’s Emotional Expression and Forms of AnthropomorphismArchives of Design Research10.15187/adr.2022.05.35.2.13735:2(137-153)Online publication date: 31-May-2022
https://doi.org/10.15187/adr.2022.05.35.2.137
Kaur SSharma R(2021)Emotion AI: Integrating Emotional Intelligence with Artificial Intelligence in the Digital WorkplaceInnovations in Information and Communication Technologies (IICT-2020)10.1007/978-3-030-66218-9_39(337-343)Online publication date: 16-Jul-2021
https://doi.org/10.1007/978-3-030-66218-9_39
Show More Cited By

Index Terms

Communicative facial displays as a new conversational modality
1. Computing methodologies
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI theory, concepts and models
    2. Interaction paradigms
      1. Natural language interfaces
  2. Interaction design
    1. Interaction design theory, concepts and paradigms

Recommendations

Using a human face in an interface
CHI '94: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Authoring Communicative Behaviors for Situated, Embodied Characters
ICMI '14: Proceedings of the 16th International Conference on Multimodal Interaction

Embodied conversational agents hold great potential as multimodal interfaces due to their ability to communicate naturally using speech and nonverbal cues. The goal of my research is to enable animators and designers to endow ECAs with interactive ...
Speech dialogue with facial displays: multimodal human-computer conversation
ACL '94: Proceedings of the 32nd annual meeting on Association for Computational Linguistics

Human face-to-face conversation is an ideal model for human-computer dialogue. One of the major features of face-to-face communication is its multiplicity of communication channels that act on multiple modalities. To realize a natural multimodal ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '93: Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems

May 1993

547 pages

ISBN:0897915755

DOI:10.1145/169059

Chairmen:
Bert Arnold
Delft Univ. of Technology, The Netherlands
,
Gerrit van der Veer
Univ. of Twente and Vrije Univ., The Netherlands
,
Ted White
Univ. of Twente and Univ. of Ultrecht, Utrecht, The Netherlands

Copyright © 1993 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCAPH: ACM SIGCAPH Computers and the Physically Handicapped
Austrian Comp Soc: Austrian Computer Society
SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques
Human Factors Soc: Human Factors Society
SIGCHI: ACM Special Interest Group on Computer-Human Interaction
SIGGROUP: ACM Special Interest Group on Supporting Group Work
IEEE-CS: Computer Society
IFIP: International Federation for Information Processing
NGI: Dutch Computer Soc - Nederlands Genoostschapvoor Informatica

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 May 1993

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

INTERCHI93

Sponsor:

SIGCAPH
Austrian Comp Soc
SIGGRAPH
Human Factors Soc
SIGCHI
SIGGROUP
IEEE-CS
IFIP
NGI

INTERCHI93: Conference on Human Factors in Computing

April 24 - 29, 1993

Amsterdam, The Netherlands

Acceptance Rates

CHI '93 Paper Acceptance Rate 62 of 330 submissions, 19%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

82
Total Citations
View Citations
907
Total Downloads

Downloads (Last 12 months)77
Downloads (Last 6 weeks)11

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Leung WWan A(2022)My Little Joy in LifeResearch Anthology on Usage, Identity, and Impact of Social Media on Society and Culture10.4018/978-1-6684-6307-9.ch057(1086-1101)Online publication date: 10-Jun-2022
https://doi.org/10.4018/978-1-6684-6307-9.ch057
Kang BKang BJun SJun S(2022)User Perception of Social Robot’s Emotional Expression and Forms of AnthropomorphismArchives of Design Research10.15187/adr.2022.05.35.2.13735:2(137-153)Online publication date: 31-May-2022
https://doi.org/10.15187/adr.2022.05.35.2.137
Kaur SSharma R(2021)Emotion AI: Integrating Emotional Intelligence with Artificial Intelligence in the Digital WorkplaceInnovations in Information and Communication Technologies (IICT-2020)10.1007/978-3-030-66218-9_39(337-343)Online publication date: 16-Jul-2021
https://doi.org/10.1007/978-3-030-66218-9_39
Leung WWan A(2019)My Little Joy in LifeImpacts of Mobile Use and Experience on Contemporary Society10.4018/978-1-5225-7885-7.ch005(70-85)Online publication date: 2019
https://doi.org/10.4018/978-1-5225-7885-7.ch005
Spatola N(2019)L’interaction Homme-Robot, de l’anthropomorphisme à l’humanisationL’Année psychologique10.3917/anpsy1.194.0515Vol. 119:4(515-563)Online publication date: 9-Dec-2019
https://doi.org/10.3917/anpsy1.194.0515
Kang XChen WKang J(2019)Art in the Age of Social Media: Interaction Behavior Analysis of Instagram Art AccountsInformatics10.3390/informatics60400526:4(52)Online publication date: 7-Dec-2019
https://doi.org/10.3390/informatics6040052
Song YQin ZKang TJin Y(2019)Robot Helps When Robot Fits: Examining the Role of Baby Robots in Fertility PromotionHealthcare10.3390/healthcare70401477:4(147)Online publication date: 15-Nov-2019
https://doi.org/10.3390/healthcare7040147
Fank JRichardson NDiermeyer F(2019)Anthropomorphising driver-truck interaction: a study on the current state of research and the introduction of two innovative conceptsJournal on Multimodal User Interfaces10.1007/s12193-019-00296-w13:2(99-117)Online publication date: 18-Mar-2019
https://doi.org/10.1007/s12193-019-00296-w
Farazdaghi EMajidzadeh FNait-Ali A(2018)Facial Rejuvenation ModelingBiometrics under Biomedical Considerations10.1007/978-981-13-1144-4_3(71-96)Online publication date: 14-Dec-2018
https://doi.org/10.1007/978-981-13-1144-4_3
Heravi FNait-Ali A(2017)A 3D dynamic shape model to simulate rejuvenation & ageing trajectory of 3D face images2017 2nd International Conference on Bio-engineering for Smart Technologies (BioSMART)10.1109/BIOSMART.2017.8095315(1-5)Online publication date: Aug-2017
https://doi.org/10.1109/BIOSMART.2017.8095315
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents