short-paper

Rapid development of multimodal interactive systems: a demonstration of platform for situated intelligence

Authors:

Dan Bohus,

Sean Andrist,

Mihai JalobeanuAuthors Info & Claims

ICMI '17: Proceedings of the 19th ACM International Conference on Multimodal Interaction

Pages 493 - 494

https://doi.org/10.1145/3136755.3143021

Published: 03 November 2017 Publication History

Get Access

Abstract

We demonstrate an open, extensible platform for developing and studying multimodal, integrative-AI systems. The platform provides a time-aware, stream-based programming model for parallel coordinated computation, a set of tools for data visualization, processing, and learning, and an ecosystem of pluggable AI components. The demonstration will showcase three applications built on this platform and highlight how the platform can significantly accelerate development and research in multimodal interactive systems.

References

[1]

2017. Platform for Situated Intelligence. http://aka.ms/psi. (2017).

Google Scholar

[2]

Itseez. 2015. Open Source Computer Vision Library. https://github.com/itseez/ opencv. (2015).

Google Scholar

[3]

Morgan Quigley, Brian Gerkey, Ken Conley, Josh Faust, Tully Foote, Jeremy Leibs, Eric Berger, Rob Wheeler, and Andrew Ng. 2009. ROS: an open-source Robot Operating System. In Proc. of the IEEE Intl. Conf. on Robotics and Automation (ICRA) Workshop on Open Source Robotics. Kobe, Japan.

Google Scholar

[4]

Gabriel Skantze and Samer Al Moubayed. 2012. IrisTK: a statechart-based toolkit for multi-party face-to-face interaction. In Proceedings of the 14th ACM international conference on Multimodal interaction (ICMI’12). ACM, 69–76.

Digital Library

Google Scholar

[5]

Johannes Wagner, Florian Lingenfelser, Tobias Baur, Ionut Damian, Felix Kistler, and Elisabeth André. 2013. The social signal interpretation (SSI) framework: Multimodal signal processing and recognition in real-time. In Proceedings of the 21st ACM international conference on Multimedia (MM’13). ACM, 831–834. Abstract 1 Introduction 2 Platform for Situated Intelligence 2.1 Runtime 2.2 Tools 2.3 Components 3 Demonstration Acknowledgments References

Digital Library

Google Scholar

Cited By

View all

Andrist SBohus DLi ZSoleymani M(2023)Platform for Situated Intelligence and OpenSense: A Tutorial on Building Multimodal Interactive Applications for ResearchCompanion Publication of the 25th International Conference on Multimodal Interaction10.1145/3610661.3617603(105-106)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3610661.3617603
Avenoğlu BKoeman VHindriks K(2022)A cloud-based middleware for multi-modal interaction services and applicationsJournal of Ambient Intelligence and Smart Environments10.3233/AIS-22016114:6(455-481)Online publication date: 29-Nov-2022
https://doi.org/10.3233/AIS-220161
Hartholt AMozgai S(2022)Platforms and Tools for SIA Research and DevelopmentThe Handbook on Socially Interactive Agents10.1145/3563659.3563668(261-304)Online publication date: 27-Oct-2022
https://dl.acm.org/doi/10.1145/3563659.3563668
Show More Cited By

Index Terms

Rapid development of multimodal interactive systems: a demonstration of platform for situated intelligence
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interactive systems and tools
      1. User interface toolkits

Recommendations

Demonstrating a framework for rapid development of physically situated interactive systems
HRI '19: Proceedings of the 14th ACM/IEEE International Conference on Human-Robot Interaction

We demonstrate an open, extensible framework for enabling faster development and study of physically situated interactive systems. The framework provides a programming model for parallel coordinated computation centered on temporal streams of data, a ...
Facilitating multiparty dialog with gaze, gesture, and speech
ICMI-MLMI '10: International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction

We study how synchronized gaze, gesture and speech rendered by an embodied conversational agent can influence the flow of conversations in multiparty settings. We begin by reviewing a computational framework for turn-taking that provides the foundation ...
Demonstration of a Robot Receptionist with Multi-party Situated Interaction
HRI '22: Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction

We present a demonstration of a Robot Receptionist: a situated interactive robot that can coordinate turn-taking and handle multi-party engagement and dialogue in dynamic environments, where users might enter or leave the scene at any time. We use a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ICMI '17: Proceedings of the 19th ACM International Conference on Multimodal Interaction

November 2017

676 pages

ISBN:9781450355438

DOI:10.1145/3136755

General Chairs:
Edward Lank
University of Waterloo, Canada
,
Alessandro Vinciarelli
University of Glasgow, UK
,
Program Chairs:
Eve Hoggan
Aarhus University, Denmark
,
Sriram Subramanian
University of Sussex, UK
,
Stephen A. Brewster
University of Glasgow, UK

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ICMI '17

Sponsor:

SIGCHI

ICMI '17: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

November 13 - 17, 2017

Glasgow, UK

Acceptance Rates

ICMI '17 Paper Acceptance Rate 65 of 149 submissions, 44%;

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
369
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)0

Reflects downloads up to 20 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Andrist SBohus DLi ZSoleymani M(2023)Platform for Situated Intelligence and OpenSense: A Tutorial on Building Multimodal Interactive Applications for ResearchCompanion Publication of the 25th International Conference on Multimodal Interaction10.1145/3610661.3617603(105-106)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3610661.3617603
Avenoğlu BKoeman VHindriks K(2022)A cloud-based middleware for multi-modal interaction services and applicationsJournal of Ambient Intelligence and Smart Environments10.3233/AIS-22016114:6(455-481)Online publication date: 29-Nov-2022
https://doi.org/10.3233/AIS-220161
Hartholt AMozgai S(2022)Platforms and Tools for SIA Research and DevelopmentThe Handbook on Socially Interactive Agents10.1145/3563659.3563668(261-304)Online publication date: 27-Oct-2022
https://dl.acm.org/doi/10.1145/3563659.3563668
Idowu SStrüber DBerger T(2022)Asset Management in Machine Learning: State-of-research and State-of-practiceACM Computing Surveys10.1145/354384755:7(1-35)Online publication date: 15-Dec-2022
https://dl.acm.org/doi/10.1145/3543847
Gunes HBroz FCrawford Cder Pütten AStrait MRiek L(2022)Reproducibility in Human-Robot Interaction: Furthering the Science of HRICurrent Robotics Reports10.1007/s43154-022-00094-53:4(281-292)Online publication date: 22-Oct-2022
https://doi.org/10.1007/s43154-022-00094-5
Ochoa X(2022)Multimodal Systems for Automated Oral Presentation Feedback: A Comparative AnalysisThe Multimodal Learning Analytics Handbook10.1007/978-3-031-08076-0_3(53-78)Online publication date: 31-May-2022
https://doi.org/10.1007/978-3-031-08076-0_3
Imtiaz MKennington C(2022)Incremental Unit Networks for Distributed, Symbolic Multimodal Processing and RepresentationDigital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Health, Operations Management, and Design10.1007/978-3-031-06018-2_24(344-363)Online publication date: 26-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-06018-2_24
Rowe R(2021)Representations, Affordances, and Interactive SystemsMultimodal Technologies and Interaction10.3390/mti50500235:5(23)Online publication date: 1-May-2021
https://doi.org/10.3390/mti5050023
Johnson BHernández-Cuevas BSingh UCrawford CVikas VDenham ABethel CPaiva ABroadbent EFeil-Seifer DSzafir D(2021)Towards Web-based Environments for Prototyping Social Robot ApplicationsCompanion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3434074.3447182(308-312)Online publication date: 8-Mar-2021
https://dl.acm.org/doi/10.1145/3434074.3447182
Aneja DHoegen RMcDuff DCzerwinski MKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Understanding Conversational and Expressive Style in a Multimodal Embodied Conversational AgentProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445708(1-10)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445708
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Demonstrating a framework for rapid development of physically situated interactive systems

Facilitating multiparty dialog with gaze, gesture, and speech

Demonstration of a Robot Receptionist with Multi-party Situated Interaction