research-article

Gesture and Voice-Based Natural User Interface for Electronic Whiteboard System in a Medical Emergency Department

Authors:

Alan Lopes de Sousa Freitas,

Vinícius Paes de Camargo,

Heloise Manica Paris Teixeira,

Renato Balancieri,

Thelma Elita ColanziAuthors Info & Claims

IHC '17: Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems

Article No.: 27, Pages 1 - 10

https://doi.org/10.1145/3160504.3160534

Published: 23 October 2017 Publication History

Abstract

Electronic Whiteboards are introduced in critical hospital environments, such as emergency rooms, to manage tasks developed by the multidisciplinary health team. However, the users interaction with such equipment, generally touch sensitive or by keyboard and mouse, has raised discussions about the possibility of disease transmission by hands contact. The main objective of this research was the implementation and user evaluation of a gesture and voice-based natural interaction module to the electronic whiteboard system, named SHAVI, developed for the emergency department of the University Hospital of Maringá. The use evaluation was performed with ten professionals from different health areas who work in the hospital. The evaluation method used was the observation in a controlled environment. The evaluation results has revealed the need to improve the prototype and suggest that users consider this type of interaction useful, therefore, they are interested in learning and using them.

References

[1]

Dominik Aronsky, Ian Jones, Kevin Lanaghan, and Corey M. Slovis. 2008. Supporting Patient Care in the Emergency Department with a Computerized Whiteboard System. Journal of the American Medical Informatics Association 15, 2 (mar 2008), 184--194.

[2]

Angelos Barmpoutis. 2013. Create Avatars using Kinect in Real-time. Research.Dwi.Ufl.Edu (2013). http://research.dwi.ufl.edu/ufdw/j4k/Kinect

[3]

Ann M. Bisantz, Priyadarshini R. Pennathur, Theresa K. Guarrera, Rollin J. Fairbanks, Shawna J. Perry, Frank Zwemer, and Robert L. Wears. 2010. Emergency Department Status Boards: A Case Study in Information Systems Transition. Journal of Cognitive Engineering and Decision Making 4, 1 (2010), 39--68.

[4]

Bojan Blazica. 2014. The inherent context awareness of natural user interfaces: a case study on multitouch displays. Informatica 38, 4 (2014), 385.

[5]

Elisa Boger. 2003. Electronic tracking board reduces ED patient length of stay at Indiana hospital. Journal of Emergency Nursing 29, 1 (2003), 39--43.

[6]

Simon Brooks, Rebecca Bhatia, Karen Tudge, and Ian Wilson. 2017. P-115 Implementing an electronic whiteboard system on the in patient unit. BMJ Supportive & Palliative Care 7, Suppl 1 (mar 2017), A42.3--A43.

[7]

Alana Da Gama, Pascal Fallavollita, Veronica Teichrieb, and Nassir Navab. 2015. Motor Rehabilitation Using Kinect: A Systematic Review. Games for Health Journal 4, 2 (2015), 123--135.

[8]

João Luiz de Almeida, Max Naegeler Roecker, Heloise Manica Paris Teixeira, Renato Balancieri, and Maria Madalena Dias. 2014. Um Sistema de Lousa Eletrônica para Unidade de Urgência e Emergência Médica. In CBIS - Congresso Brasileiro de Informática em Saúde. Santos.

[9]

Lars C. Ebert, Gary Hatch, Garyfalia Ampanozi, Michael J. Thali, and Steffen Ross. 2012. You Can't Touch This. Surgical Innovation 19, 3 (2012), 301--307. 22064490

[10]

Anders K. Ericsson. 2006. Protocol Analysis and Expert Thought: Concurrent Verbalizations of Thinking during Experts Performance on Representative Tasks. The Cambridge Handbook of Expertise and Expert Performance (2006), 39--68.

[11]

Luigi Gallo, Alessio Pierluigi Placitelli, and Mario Ciampi. 2011. Controller-free exploration of medical image data: Experiencing the Kinect. In 2011 24th International Symposium on Computer-Based Medical Systems (CBMS). 1--6.

Digital Library

[12]

Viviane Hasselmann, Peter Oesch, Luis Fernandez-Luque, and Stefan Bachmann. 2015. Are exergames promoting mobility an attractive alternative to conventional self-regulated exercises for elderly people in a rehabilitation setting? Study protocol of a randomized controlled trial. BMC geriatrics 15, June 2016 (2015), {1} T. Hewett, R. Baecker, S. Card, T. Carey, J. G.

[13]

Morten Hertzum and Jesper Simonsen. 2013. Work-practice changes associated with an electronic emergency department whiteboard. Health Informatics Journal 19, 1 (2013), 46--60. 23486825

[14]

Morten Hertzum and Jesper Simonsen. 2016. Effects of electronic emergency-department whiteboards on clinicians' time distribution and mental workload. Health Informatics Journal 22, 1 (2016), 3--20.

[15]

Mithun George Jacob and Juan Pablo Wachs. 2014. Context-based hand gesture recognition for the operating room. Pattern Recognition Letters 36 (2014), 196--203.

Digital Library

[16]

JSAPI. 2017. Java Speech API. (2017). https://github.com/lkuza2/java-speech-api

[17]

Cem Keskin, Koray Balci, Oya Aran, Bulent Sankur, and Lale Akarun. 2007. A Multimodal 3D Healthcare Communication System. In 2007 3DTV Conference. 1--4.

[18]

Daniel Thom Lopes, Renato Balancieri, Heloise Manica, Paris Teixeira, and Maria Madalena Dias. 2014. Electronic Whiteboard in Hospitals: a systematic review. 6, 4 (2014), 166--171.

[19]

Rajshri Mainthia, Timothy Lockney, Alexandr Zotov, Daniel J. France, Marc Bennett, Paul J. St. Jacques, William Furman, Stephanie Randa, Nancye Feistritzer, Roland Eavey, Susie Leming-Lee, and Shilo Anders. 2012. Novel use of electronic whiteboard in the operating room increases surgical team compliance with pre-incision safety practices. Surgery 151, 5 (2012), 660--666.

[20]

L.J. Brewster Mallalieu, A. Sharma, A. Jamshidi, Y. Cao, A. Kapur, J. Pinsky, J. Mogavero, and L. Potters. 2011. A Virtual Whiteboard for Improvement of Coordination of Physics Processes in a Multi-site Radiation Therapy Department. International Journal of Radiation Oncology*Biology*Physics 81, 2 (2011), S697. Proceedings of the American Society for Radiation Oncology 53rd Annual Meeting

[21]

Microsoft. 2013a. Skeletal Tracking. (2013). https://msdn.microsoft.com/en-us/library/hh973074.aspx

[22]

Microsoft. 2013b. Tracking Users with Kinect Skeletal Tracking. (2013). https://msdn.microsoft.com/en-us/library/jj131025.aspx

[23]

Hossein Mousavi Hondori and Maryam Khademi. 2014. A Review on Technical and Clinical Impact of Microsoft Kinect on Physical Therapy and Rehabilitation. Journal of Medical Engineering 2014 (2014), {1} T. Hewett, R. Baecker, S. Card, T. Carey, J. G.

[24]

Tara Mullaney, Björn Yttergren, and Erik Stolterman. 2013. Positional acts: using a Kinect™ sensor to reconfigure patient roles within radiotherapy treatment. Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction - TEI '14 (2013), {1} T. Hewett, R. Baecker, S. Card, T. Carey, J. G.

Digital Library

[25]

Jakob Nielsen and Thomas K. Landauer. 1993. A Mathematical Model of the Finding of Usability Problems. In Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems (CHI '93). ACM, New York, NY, USA, 206--213.

Digital Library

[26]

Julien Pauchot, Laetitia Di Tommaso, Ahmed Lounis, Mourad Benassarou, Pierre Mathieu, Dominique Bernot, and Sébastien Aubry. 2015. Leap Motion Gesture Control With Carestream Software in the Operating Room to Control Imaging. Surgical Innovation 22, 6 (2015), 615--620. 26002115

[27]

Mauricio Pradella. 2011. Design de interação: aplicação de interfaces intangíveis em simulações para o aprendizado a distância. Dissertação de Mestrado. Universidade Anhembi Morumbi.

[28]

Guillermo M. Rosa and María L. Elizondo. 2014. Use of a gesture user interface as a touchless image navigation system in dental surgery: Case series report. Imaging Science in Dentistry 44, 2 (2014), {1} T. Hewett, R. Baecker, S. Card, T. Carey, J. G.

[29]

Daniel Sonntag, Christian Husodo Schulz, Christian Reuschling, and Luis Galarraga. 2012. RadSpeech, a mobile dialogue system for radiologists. Proceedings of the International Conference on Intelligent User Interfaces (IUI). International Conference on Intelligent User Interfaces (IUI), 17th, February 14-17, Lisboa, Portugal (2012), {1} T. Hewett, R. Baecker, S. Card, T. Carey, J. G.

Digital Library

[30]

World Health Organisation (WHO). 2009. WHO Guidelines on Hand Hygiene in Health Care: First Global Patient Safety Challenge Clean Care Is Safer Care. World Health 30, 1 (2009), 270.

Cited By

Da Costa Nunes ERibeiro GMonteiro IGonçalves E(2023)Digital Accessibility at the Brazilian Symposium on Human Factors in Computing Systems (IHC): An Updated Systematic Literature ReviewProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638075(1-15)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3638067.3638075
Guo SZhan LCao YZheng CZhou GGong J(2023)Touch-and-HealProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35962587:2(1-33)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3596258
Murad CCandello HMunteanu C(2023)What’s The Talk on VUI Guidelines? A Meta-Analysis of Guidelines for Voice User Interface DesignProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3597129(1-16)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3571884.3597129
Show More Cited By

Index Terms

Gesture and Voice-Based Natural User Interface for Electronic Whiteboard System in a Medical Emergency Department
1. Applied computing
  1. Life and medical sciences
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Web-based database and SMS to facilitate healthcare medical emergency
HIKM '11: Proceedings of the Fourth Australasian Workshop on Health Informatics and Knowledge Management - Volume 120

Healthcare and medical emergency are essential systems in human life; so that many countries work toward having it. Investigation of Malaysian case shows that it suffer from locality, missing of unified electronic medical record EMR and lack of ...
Situation role-based privacy control using dynamic credentials for emergency health services

Emerging Health Information Technologies (HIT) such as Electronic Health Record (EHR) and PHR systems and Heath Information Exchanges (HIEs) facilitate access to and sharing of patients' medical data distributed in different HIT systems. One of the many ...
Building a National Electronic Medical Record Exchange System - Experiences in Taiwan

Electronic medical record (EMR) can support a secure, real-time, point-of-care, patient centric information resource for clinical care.Taiwan's government has been promoting the EMR adoption since 2000.We describe the EMR adoption strategies, current ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

IHC '17: Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems

October 2017

622 pages

ISBN:9781450363778

DOI:10.1145/3160504

Conference Chairs:
Isabela Gasparini
UDESC
,
Lara S. G. Piccolo
The Open University
,
Program Chairs:
Luciana A. M. Zaina
UFSCar
,
Roberto Pereira
UFPR

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

UDESC: Santa Catarina State University
Springer
SBC: Sociedade Brasileira de Computação
ACM: Association for Computing Machinery
UFSCar: Federal University of São Carlos
CAPES: Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
UFPR: Universidade Federal do Paraná
CNPq: Conselho Nacional de Desenvolvimento Cientifico e Tecn
CGIBR: Comite Gestor da Internet no Brazil
OU: The Open University

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

IHC 2017

IHC 2017: Brazilian Symposium on Human Factors in Computing Systems

October 23 - 27, 2017

Joinville, Brazil

Acceptance Rates

IHC '17 Paper Acceptance Rate 66 of 184 submissions, 36%;

Overall Acceptance Rate 331 of 973 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
146
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Da Costa Nunes ERibeiro GMonteiro IGonçalves E(2023)Digital Accessibility at the Brazilian Symposium on Human Factors in Computing Systems (IHC): An Updated Systematic Literature ReviewProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638075(1-15)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3638067.3638075
Guo SZhan LCao YZheng CZhou GGong J(2023)Touch-and-HealProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35962587:2(1-33)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3596258
Murad CCandello HMunteanu C(2023)What’s The Talk on VUI Guidelines? A Meta-Analysis of Guidelines for Voice User Interface DesignProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3597129(1-16)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3571884.3597129
Hirsch LLi JMayer SButz A(2022)A Survey of Natural Design for InteractionProceedings of Mensch und Computer 202210.1145/3543758.3543773(240-254)Online publication date: 4-Sep-2022
https://dl.acm.org/doi/10.1145/3543758.3543773
Kimura KMoriya KGuerino GTeixeira HMonteiro Ida Hora Rodrigues Kde Gois Ribeiro Darin TFreire AMota M(2021)Gesture-based Interaction Systems in Hospital Critical EnvironmentProceedings of the XX Brazilian Symposium on Human Factors in Computing Systems10.1145/3472301.3484350(1-7)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3472301.3484350
de Camargo VBalancieri RTeixeira HGuerino GMonteiro Ida Hora Rodrigues Kde Gois Ribeiro Darin TFreire AMota M(2021)Touchless Modalities of Human-Computer Interaction in HospitalsProceedings of the XX Brazilian Symposium on Human Factors in Computing Systems10.1145/3472301.3484328(1-11)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3472301.3484328

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents