Speech and Sketching: An Empirical Study of Multimodal Interaction

Files

083-090.pdf (584.31 KB)

Date

2007

Authors

Adler, A.
Davis, R.

Publisher

The Eurographics Association

Abstract

Sketch recognition can capture the sketching component of a multimodal conversation about design, but it does not capture information conveyed in the other modalities. The informal speech that accompanies a sketch often has a considerable amount of additional information. We want to develop a digital whiteboard capable of understanding both sketching and speech, and capable of participating in a conversation similar to one that the user would have with a human design partner. We conducted a user study to help us understand what kinds of conversations users would have with a whiteboard capable of recognizing a sketch. We report results that we believe will help guide the design of an effective multimodal interface, and discuss implications for system architectures.

        @inproceedings{:10.2312/SBM/SBM07/083-090
,
booktitle = {EUROGRAPHICS Workshop on Sketch-Based Interfaces and Modeling
},
editor = {Michiel van de Panne and Eric Saund
},
title = {{Speech and Sketching: An Empirical Study of Multimodal Interaction
}},
author = {Adler, A. and 
Davis, R.
},
year = {2007
},
publisher = {The Eurographics Association
},
ISSN = {1812-3503
},
ISBN = {978-3-905674-00-2
},
DOI = {/10.2312/SBM/SBM07/083-090
}
}