research-article

Multimodal interaction with speech and physical touch interface in a media center application

Authors:

Markku Turunen,

Aleksi Kallinen,

Ivàn Sànchez,

Juha-Pekka Rajaniemi,

Jaakko Hakulinen,

Pellervo Valkama,

Toni Miettinen,

Mikko Pyykkönen,

Timo Saloranta,

Ekaterina Gilman,

Roope RaisamoAuthors Info & Claims

ACE '09: Proceedings of the International Conference on Advances in Computer Entertainment Technology

Pages 19 - 26

https://doi.org/10.1145/1690388.1690392

Published: 29 October 2009 Publication History

Abstract

We present a multimodal media center interface based on a novel combination of new modalities. The application is based on a combination of a large high-definition display and a mobile phone. Users can interact with the system using speech input (speech recognition), physical touch (touching physical icons with the mobile phone), and gestures. We present the key results from a laboratory experiment where user expectations and actual usage experiences are compared.

References

[1]

Ailisto H, Pohjanheimo L, Välkkynen P, Strömmer E, Tuomisto T and Korhonen I (2006) Bridging the physical and virtual worlds by local connectivity-based physical selection, Personal Ubiquitous Computing, 10(6)333--344.

Digital Library

[2]

Arhippainen, L. and Tähti, M. (2003) Empirical Evaluation of User Experience in Two Adaptive Mobile Application Prototypes, in Proc. of MUM 2003, Norrköping, Sweden, pp. 27--34.

[3]

Battarbee, K. and Koskinen, I. Co-experience: user experience as interaction. CoDesign, 2005. 1(1), pp. 5--18.

[4]

Bederson, B. B. 2000. Fisheye menus. In Proceedings of the 13th Annual ACM Symposium on User interface Software and Technology (San Diego, California, United States, November 06--08, 2000). UIST '00. ACM, New York, NY, 217--225.

Digital Library

[5]

Bederson, B. B., Clamage, A., Czerwinski, M. P., and Robertson, G. G. 2004. DateLens: A fisheye calendar interface for PDAs. ACM Trans. Comput.-Hum. Interact. 11, 1 (Mar. 2004), 90--119.

Digital Library

[6]

Bederson, B. B., Grosjean, J., and Meyer, J. 2004. Toolkit Design for Interactive Structured Graphics. IEEE Trans. Softw. Eng. 30, 8 (Aug. 2004), 535--546.

Digital Library

[7]

Berglund, A., and Qvarfordt, P. 2003. Error Resolution Strategies for Interactive Television Speech Interfaces. In Proceedings of International Conference on Human-Computer Interaction (INTERACT '03). IFIP, Amsterdam, 2003, 105--112.

[8]

Broll G, Siorpaes S, Rukzio E, Paolucci M, Haamard J, Wagner M and Schmidt A (2007) Supporting Mobile Services Usage through Physical Mobile Interaction. Proc 5th IEEE Intl Conf on Pervasive Computing and Communications. White Plains, NY, USA, pp. 262--271.

Digital Library

[9]

Ferscha, A., Vogl, S., Emsenhuber, B., and Wally, B. 2007. Physical shortcuts for media remote controls. In Proceedings of the 2nd international Conference on intelligent Technologies For interactive Entertainment. ICST, Brussels, Belgium, 1--8.

Digital Library

[10]

Hassenzahl, M., Burmester, M, and Koller F. 2003. Attrak-Diff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In J. Ziegler & G. Szwillus (Hrsg.), Mensch & Computer 2003. Interaktion in Bewegung. Stuttgart, Leipzig: B. G. Teubner, 187--196.

[11]

Hassenzahl, M. 2004. The thing and I: understanding the relationship between user and product. In Funology: From Usability To Enjoyment, M. A. Blythe, K. Overbeeke, A. F. Monk, and P. C. Wright, Eds. Kluwer Academic Publishers, Norwell, MA, 31--42.

Digital Library

[12]

Hassenzahl, M. The interplay of beauty, goodness, and usability in interactive products. Human-Computer Interaction, 2004. 19(4), pp. 319--349.

Digital Library

[13]

Hassenzahl, M. and Tractinsky, N. (2006) User Experience -- A Research Agenda. Behaviour and Information Technology, Vol. 25, No. 2, pp. 91--97.

[14]

Ibrahim, A., and Johansson, P. 2003. Multimodal Dialogue Systems: A Case Study for Interactive TV. Carbonell, Noelle; Stephanidis, Constantine (Eds.) Universal Access. Theoretical Perspectives, Practice, and Experience, 7th ERCIM International Workshop on User Interfaces for All, Revised Papers. Springer, LNCS, Vol. 2615. 209--218.

Digital Library

[15]

Kankainen, A. (2003) UCPCD: User-Centered Product Concept Design, in Proc. of the 2003 conference on Designing for user experiences, ACM Press, pp. 1--13.

Digital Library

[16]

NFC Forum. Near Field Communication and the NFC Forum: The Keys to Truly Interoperable Communications. http://www.nfc-forum.org/resources/white_papers (Last Revised16.06.09).

[17]

NFC Forum. NFC Data Exchange Format (NDEF). Accessible from http://www.nfc-forum.org/specs/. Last access 16-06-09.

[18]

Riekki J., Sànchez I., and Pyykkönen M. Universal remote control for the smart world. In Proceedings of 5th International Conference on Ubiquitous Intelligence and Computing, UIC 2008, pages 563--577, Oslo, Norway, June 23--25 2008.

Digital Library

[19]

Sànchez I, Riekki J, Pyykkönen M (2008) Touch & Control: Interacting with Services by Touching RFID Tags. In Proceedings of the 2nd International Workshop on RFID Technology - Concepts, Applications, Challenges (IWRT 2008), In conjunction with ICEIS 2008. Barcelona, Spain, June 12--13, 2008. pp 53--62. http://www.ee.oulu.fi/research/isg/publications/ID/1272.

[20]

Schlömer, T., Poppinga, B., Henze, N., and Boll, S. 2008. Gesture recognition with a Wii controller. In Proceedings of the 2nd international Conference on Tangible and Embedded interaction (Bonn, Germany, February 18--20, 2008). TEI '08. ACM, New York, NY, 11--14.

Digital Library

[21]

Soronen H, Turunen M, Hakulinen J. 2008. Voice Commands in Home Environment - a Consumer Survey, In Proceedings of Interspeech 2008: 2078--2081.

[22]

Turunen, M., Melto, A., Hakulinen, J., Kainulainen, A., and Heimonen, T. User Expectations, User Experiences and Objective Metrics in a Multimodal Mobile Application. Proceedings of the Third Workshop on Speech in Mobile and Pervasive Environments, 2008.

[23]

Turunen, M., Hakulinen, J., Melto, A., Heimonen, T., Laivo, T., and Hella, J. SUXES -- User Experience Evaluation Method for Spoken and Multimodal Interaction. In Proceedings of Interspeech 2009.

[24]

Turunen, M., Hakulinen, J., Melto, A., Hella, J., Rajaniemi, J.-P., Mäkinen, E., Rantala, J., Heimonen, T., Laivo, T., Soronen, H., Hansen, M., Valkama, P. Miettinen, T., Raisamo, R. Speech-based and Multimodal Media Center for Different User Groups. In Proceedings of Interspeech 2009.

[25]

Wittenburg, K., Lanning, T., Schwenke, D., Shubin, H., and Vetro, A. 2006. The prospects for unrestricted speech input for TV content search. In Proceedings of the Working Conference on Advanced Visual interfaces (Venezia, Italy, May 23--26, 2006). AVI '06. ACM, New York, NY, 352--359.

Digital Library

Cited By

Gao WDu KLuo YShi WYu CShi Y(2024)EasyAsk: An In-App Contextual Tutorial Search Assistant for Older Adults with Voice and Touch InputsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785168:3(1-27)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678516
Steinhaeusser SLein MDonnermann MLugrin B(2022)Designing Social Robots' Speech in the Hotel Context - A Series of Online Studies2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN53752.2022.9900668(163-170)Online publication date: 29-Aug-2022
https://dl.acm.org/doi/10.1109/RO-MAN53752.2022.9900668
Barreto Fde Abreu RJosué MMontevecchi EValentim PMuchaluat-Saade D(2022)Providing multimodal and multi-user interactions for digital tv applicationsMultimedia Tools and Applications10.1007/s11042-021-11847-382:4(4821-4846)Online publication date: 18-Jul-2022
https://doi.org/10.1007/s11042-021-11847-3
Show More Cited By

Index Terms

Multimodal interaction with speech and physical touch interface in a media center application
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens
    2. Interaction paradigms

Recommendations

User experience of speech controlled media center for physically disabled users
MindTrek '09: Proceedings of the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era

In this paper, we present results from a long-term user pilot study of speech controlled media center. The pilot users in this case were physically disabled and the system was installed in their apartment for six weeks. We designed a multimodal media ...
Accessible Multimodal Media Center Application for Blind and Partially Sighted People
Theoretical and Practical Computer Applications in Entertainment

We present a multimodal media center interface designed for blind and partially sighted people. It features a zooming focus-plus-context graphical user interface coupled with speech output and haptic feedback. A multimodal combination of gestures, key ...
Design and evaluation of techniques for mobile interaction with dynamic NFC-displays
TEI '11: Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction

Near Field Communication (NFC) is a technology for mobile, touch-based interaction with tagged objects that can serve as physical user interfaces (UI). Dynamic NFC-displays use a grid of NFC-tags as a physical UI and combine it with a projected ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ACE '09: Proceedings of the International Conference on Advances in Computer Entertainment Technology

October 2009

456 pages

ISBN:9781605588643

DOI:10.1145/1690388

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Foundation of the Hellenic World

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 October 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ACE '09

Sponsor:

ACE '09: International Conference in Advances in Computer Entertainment Technology

October 29 - 31, 2009

Athens, Greece

Acceptance Rates

Overall Acceptance Rate 36 of 90 submissions, 40%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
735
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gao WDu KLuo YShi WYu CShi Y(2024)EasyAsk: An In-App Contextual Tutorial Search Assistant for Older Adults with Voice and Touch InputsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785168:3(1-27)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678516
Steinhaeusser SLein MDonnermann MLugrin B(2022)Designing Social Robots' Speech in the Hotel Context - A Series of Online Studies2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN53752.2022.9900668(163-170)Online publication date: 29-Aug-2022
https://dl.acm.org/doi/10.1109/RO-MAN53752.2022.9900668
Barreto Fde Abreu RJosué MMontevecchi EValentim PMuchaluat-Saade D(2022)Providing multimodal and multi-user interactions for digital tv applicationsMultimedia Tools and Applications10.1007/s11042-021-11847-382:4(4821-4846)Online publication date: 18-Jul-2022
https://doi.org/10.1007/s11042-021-11847-3
Leonidis AKorozi MKouroumalis VPoutouris EStefanidi EArampatzis DSykianaki EAnyfantis NKalligiannakis ENicodemou VStefanidi ZAdamakis EStivaktakis NEvdaimon TAntona M(2019)Ambient Intelligence in the Living RoomSensors10.3390/s1922501119:22(5011)Online publication date: 16-Nov-2019
https://doi.org/10.3390/s19225011
Vuorio JOkkonen JViteli J(2019)User Expectations and Experiences in Using Location-Based Game in Educational ContextDigital Turn in Schools—Research, Policy, Practice10.1007/978-981-13-7361-9_2(17-35)Online publication date: 5-Jun-2019
https://doi.org/10.1007/978-981-13-7361-9_2
Vuorio JOkkonen JViteli J(2018)Finnish Upper Secondary Students User Expectations and Experiences Using MALL SystemProceedings of the 22nd International Academic Mindtrek Conference10.1145/3275116.3275150(236-243)Online publication date: 10-Oct-2018
https://dl.acm.org/doi/10.1145/3275116.3275150
Anyfantis NKalligiannakis ETsiolkas ALeonidis AKorozi MLilitsis PAntona MStephanidis C(2018)AmITVProceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference10.1145/3197768.3201548(507-514)Online publication date: 26-Jun-2018
https://dl.acm.org/doi/10.1145/3197768.3201548
Nguyen NLee H(2018)SocioCon: A Social Circle for Your Interactive DevicesDesign, User Experience, and Usability: Designing Interactions10.1007/978-3-319-91803-7_47(623-639)Online publication date: 15-Jul-2018
https://dl.acm.org/doi/10.1007/978-3-319-91803-7_47
Lee JLee CKim G(2017)Vouch: multimodal touch-and-voice input for smart watches under difficult operating conditionsJournal on Multimodal User Interfaces10.1007/s12193-017-0246-y11:3(289-299)Online publication date: 19-Jun-2017
https://doi.org/10.1007/s12193-017-0246-y
Mentor D(2015)From App Attack to Goal-Oriented Tablet UseTablets in K-12 Education10.4018/978-1-4666-6300-8.ch001(1-21)Online publication date: 2015
https://doi.org/10.4018/978-1-4666-6300-8.ch001
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents