Article

A look under the hood: design and development of the first SmartWeb system demonstrator

Authors:

Norbert Reithinger,

Simon Bergweiler,

Norbert Pfleger,

Massimo Romanelli,

Daniel SonntagAuthors Info & Claims

ICMI '05: Proceedings of the 7th international conference on Multimodal interfaces

Pages 159 - 166

https://doi.org/10.1145/1088463.1088492

Published: 04 October 2005 Publication History

Abstract

Experience shows that decisions in the early phases of the development of a multimodal system prevail throughout the life-cycle of a project. The distributed architecture and the requirement for robust multimodal interaction in our project SmartWeb resulted in an approach that uses and extends W3C standards like EMMA and RDFS. These standards for the interface structure and content allowed us to integrate available tools and techniques. However, the requirements in our system called for various extensions, e.g., to introduce result feedback tags for an extended version of EMMA. The interconnection framework depends on a commercial telephone voice dialog system platform for the dialog-centric components while the information access processes are linked using web service technology. Also in the area of this underlying infrastructure, enhancements and extensions were necessary. The first demonstration system is operable now and will be presented at the Football World Cup 2006 in Germany.

References

[1]

J. Alexandersson and T. Becker. The Formal Foundations Underlying Overlay. In Proceedings of the Fifth International Workshop on Computational Semantics (IWCS-5), Tilburg, The Netherlands, 2003.

[2]

J. Alexandersson, T. Becker, R. Engel, M. Löckelt, E. Pecourt, P. Poller, N. Pfleger, and N. Reithinger. Ends-based dialogue processing. In Proceedings, Second International Workshop on Scalable Natural Language Understanding, Boston, 2004.

[3]

J. Alexandersson, R. Engel, M. Kipp, S. Koch, U. Küssner, N. Reithinger, and M. Stede. Modeling negotiation dialogs. In W. Wahlster, editor, Verbmobil: Foundations of Speech-to-Speech Translation, pages 441 -- 451. Springer, 2000.

[4]

R. Amant and C. Healey. Usability guidelines for interactive search in direct manipulation systems. In Proceedings of The International Joint Conference on Artificial Intelligence (IJCAI), 2001.

Digital Library

[5]

K. Bontcheva, V. Tablan, D. Maynard, and H. Cunningham. Evolving GATE to Meet New Challenges in Language Engineering. Natural Language Engineering, 10, 2004.

Digital Library

[6]

A. J. Cheyer and D. L. Martin. The Open Agent Architecture. Autonomous Agents and Multi-Agent Systems, 4(1--2):143--148, 2001.

Digital Library

[7]

R. Engel. SPIN: Language understanding for spoken dialogue systems using a production system approach. In Proceedings of 7th International Conference on Spoken Language Processing (ICSLP-2002), pages 2717--2720, Denver, Colorado, USA, 2002.

[8]

D. Fensel, J. A. Hendler, H. Lieberman, and W. Wahlster, editors. Spinning the Semantic Web: Bringing the World Wide Web to Its Full Potential. MIT Press, 2003.

Digital Library

[9]

L. Gorlenko and R. Merrick. No wires attached: Usability challenges in the connected mobile world. IBM Systems Journal, 2003.

Digital Library

[10]

N. Guarino and C. A. Welty. An overview of ontoclean. In S. Staab and R. Studer, editors, Handbook on Ontologies, International Handbooks on Information Systems, pages 151--172. Springer, 2004.

[11]

G. Herzog, A. Ndiaye, S. Merten, H. Kirchmann, T. Becker, and P. Poller. Large-scale Software Integration for Spoken Language and Multimodal Dialog Systems. Natural Language Engineering, 10(2/4):283--305, 2004.

Digital Library

[12]

G. E. Krasner and S. T. Pope. A cookbook for using the model-view controller user interface paradigm in smalltalk-80. J. Object Oriented Program., 1(3):26--49, 1988.

Digital Library

[13]

R. Longoria, editor. Designing Software for the Mobile Context. A Practitioner's Guide. Springer, 2004.

Digital Library

[14]

M. T. Maybury and W. Wahlster, editors. Readings in Intelligent User Interfaces. Morgan Kaufmann, San Francisco, 1998.

Digital Library

[15]

S. Oviatt. Ten myths of multimodal interaction. Communications of the ACM, 42(11):74--81, 1999.

Digital Library

[16]

N. Reithinger, J. Alexandersson, T. Becker, A. Blocher, R. Engel, M. Löckelt, J. Müller, N. Pfleger, P. Poller, M. Streit, and V. Tschernomas. SmartKom: Adaptive and Flexible Multimodal Access to Multiple Applications. In Proc. of the 5th Int. Conf. on Multimodal Interfaces, pages 101--108, Vancouver, Canada, 2003. ACM Press.

Digital Library

[17]

N. Reithinger, D. Fedeler, A. Kumar, C. Lauer, E. Pecourt, and L. Romary. MIAMM - A Multimodal Dialogue System Using Haptics. In J. van Kuppevelt, L. Dybkjaer, and N. O. Bernsen, editors, Advances in Natural Multimodal Dialogue Systems. Springer, 2005.

[18]

S. Seneff, R. Lau, and J. Polifroni. Organization, Communication, and Control in the Galaxy-II Conversational System. In Proc. of Eurospeech'99, pages 1271--1274, Budapest, Hungary, 1999.

[19]

K. R. Thorisson, C. Pennock, T. List, and J. DiPirro. Artificial intelligence in computer graphics: A constructionist approach. Computer Graphics, pages 26--30, February 2004.

Digital Library

[20]

W. Wahlster, editor. VERBMOBIL: Foundations of Speech-to-Speech Translation. Springer, 2000.

[21]

W. Wahlster. Smartweb: Mobile applications of the semantic web. In P. Dadam and M. Reichert, editors, GI Jahrestagung 2004, pages 26--27. Springer, 2004.

Cited By

Barz MBhatti OLüers BPrange ASonntag D(2021)Multisensor-Pipeline: A Lightweight, Flexible, and Extensible Framework for Building Multimodal-Multisensor InterfacesCompanion Publication of the 2021 International Conference on Multimodal Interaction10.1145/3461615.3485432(13-18)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3461615.3485432
Kumar VJoshi SKando NSakai TJoho HLi Hde Vries AWhite R(2017)Incomplete Follow-up Question Resolution using Retrieval based Sequence to Sequence LearningProceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3077136.3080801(705-714)Online publication date: 7-Aug-2017
https://dl.acm.org/doi/10.1145/3077136.3080801
Sonntag D(2011)Intuition as instinctive dialogueComputing with instinct10.5555/1980745.1980753(82-106)Online publication date: 1-Jan-2011
https://dl.acm.org/doi/10.5555/1980745.1980753
Show More Cited By

Index Terms

A look under the hood: design and development of the first SmartWeb system demonstrator
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Graphics input devices
    2. Interaction paradigms
      1. Natural language interfaces

Recommendations

Multimodal spatial reference in mediated environments: users' preferences and the pragmatics of pointing and talking
CHI EA '06: CHI '06 Extended Abstracts on Human Factors in Computing Systems

This paper describes the current results and future developments of a project on multimodal spatial reference in mediated environments. The database consists of video-recorded sessions, with 120 participants in three experimental designs, contrasting ...
Usability framework for the design and evaluation of multimodal interaction
BCS-HCI '08: Proceedings of the 22nd British HCI Group Annual Conference on People and Computers: Culture, Creativity, Interaction - Volume 2

The design and evaluation of multimodal interaction is difficult. For designers in industry, developing multimodal interaction systems is a big challenge. Although past researches have presented various methodologies, they have addressed only specific ...
Speech and face-to-face communication - An introduction

This issue focuses on face-to-face speech communication. Research works have demonstrated that this communicative situation is essential to language acquisition and development (e.g. naming). Face-to-face communication is in fact much more than speaking ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '05: Proceedings of the 7th international conference on Multimodal interfaces

October 2005

344 pages

ISBN:1595930280

DOI:10.1145/1088463

General Chairs:
Gianni Lazzari
ITC-irst, Trento (Italy)
,
Fabio Pianesi
ITC-irst, Trento (Italy)
,
Program Chairs:
James Crowley
I.N.P. Grenoble (France)
,
Kenji Mase
Nagoya University (Japan)
,
Sharon Oviatt
Oregon Health & Sciences University

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 October 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICMI05

Sponsor:

ICMI05: Seventh International Conference on Multimodal Interfaces 2005

October 4 - 6, 2005

Torento, Italy

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
510
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)1

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Barz MBhatti OLüers BPrange ASonntag D(2021)Multisensor-Pipeline: A Lightweight, Flexible, and Extensible Framework for Building Multimodal-Multisensor InterfacesCompanion Publication of the 2021 International Conference on Multimodal Interaction10.1145/3461615.3485432(13-18)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3461615.3485432
Kumar VJoshi SKando NSakai TJoho HLi Hde Vries AWhite R(2017)Incomplete Follow-up Question Resolution using Retrieval based Sequence to Sequence LearningProceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3077136.3080801(705-714)Online publication date: 7-Aug-2017
https://dl.acm.org/doi/10.1145/3077136.3080801
Sonntag D(2011)Intuition as instinctive dialogueComputing with instinct10.5555/1980745.1980753(82-106)Online publication date: 1-Jan-2011
https://dl.acm.org/doi/10.5555/1980745.1980753
Sonntag D(2011)Intuition as Instinctive DialogueComputing with Instinct10.1007/978-3-642-19757-4_6(82-106)Online publication date: 2011
https://doi.org/10.1007/978-3-642-19757-4_6
van Schooten Bop den Akker R(2011)Vidiam: Corpus-based Development of a Dialogue Manager for Multimodal Question AnsweringInteractive Multi-modal Question-Answering10.1007/978-3-642-17525-1_3(25-56)Online publication date: 8-Apr-2011
https://doi.org/10.1007/978-3-642-17525-1_3
Berardinelli LCortellessa VDi Marco A(2010)A Profile-Driven Environment for Modeling and Analyzing Context-Aware Software ServicesProceedings of the 2010 36th EUROMICRO Conference on Software Engineering and Advanced Applications10.1109/SEAA.2010.47(199-208)Online publication date: 1-Sep-2010
https://dl.acm.org/doi/10.1109/SEAA.2010.47
Johnston MCrowley JIvanov YWren CGatica-Perez DJohnston MStiefelhagen R(2009)Building multimodal applications with EMMAProceedings of the 2009 international conference on Multimodal interfaces10.1145/1647314.1647325(47-54)Online publication date: 2-Nov-2009
https://dl.acm.org/doi/10.1145/1647314.1647325
Van schooten BOp den akker RRosset SGalibert OMax AIllouz G(2009)Follow-up question handling in the imix and ritel systemsNatural Language Engineering10.1017/S135132490800492015:1(97-118)Online publication date: 1-Jan-2009
https://dl.acm.org/doi/10.1017/S1351324908004920
Batliner AHacker CNöth E(2009)To talk or not to talk with a computerJournal on Multimodal User Interfaces10.1007/s12193-009-0016-62:3-4(171-186)Online publication date: 26-Aug-2009
https://doi.org/10.1007/s12193-009-0016-6
Perritaz DSalzmann CGillet DNaef OBapst JBarras FMugellini EAbou Khaled O(2009)6th Sense--- Toward a Generic Framework for End-to-End Adaptive Wearable Augmented RealityHuman Machine Interaction10.1007/978-3-642-00437-7_11(280-310)Online publication date: 27-Mar-2009
https://dl.acm.org/doi/10.1007/978-3-642-00437-7_11
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents