research-article

Single operator, multiple robots: an eye movement based theoretic model of operator situation awareness

Authors:

Raj M. Ratwani,

J. Malcolm McCurry,

J. Gregory TraftonAuthors Info & Claims

HRI '10: Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction

Pages 235 - 242

Published: 02 March 2010 Publication History

Abstract

For a single operator to effectively control multiple robots, operator situation awareness is a critical component of the human-robot system. There are three levels of situation awareness: perception, comprehension, and projection into the future [1]. We focus on the perception level to develop a theoretic model of the perceptual-cognitive processes underlying situation awareness. Eye movement measures were developed as indicators of cognitive processing and these measures were used to account for operator situation awareness on a supervisory control task. The eye movement based model emphasizes the importance of visual scanning and attention allocation as the cognitive processes that lead to operator situation awareness and the model lays the groundwork for real-time prediction of operator situation awareness.

References

[1]

M. R. Endsley, "Toward a theory of situation awareness in dyanmic systems," Human Factors, vol. 37, pp. 32--64, 1995.

[2]

J. W. Crandall, M. A. Goodrich, D. R. Olsen, and C. W. Nielsen, "Validating human-robot interaction schemes in multitasking environments," Systems, Man, and Cybernetics, vol. 35, pp. 438--449, 2005.

Digital Library

[3]

M. L. Cummings and P. J. Mitchell, "Predicting controller capacity in supervisory control of multiple UAVs," IEEE Systems, Man, and Cybernetcis, Part A: Systems and Humans, vol. 38, pp. 451--460, 2008.

Digital Library

[4]

R. Parasuraman, T. B. Sheridan, and C. D. Wickens, "Situation awareness, mental workload, and trust in automation: Viable, empirically supported cogntive engineering constructs," Journal of Cognitive Engineering and Decision Making, vol. 2, pp. 140--160, 2008.

[5]

R. M. Taylor, "Situational Awareness rating technique (SART): The development of a tool for aircrew systems design," in AGARD Conference Proceedings, 1990, pp. 3/1--3/17.

[6]

M. R. Endsley, "Measurement of situation awareness in dynamic systems.," Human Factors, vol. 37, pp. 65--84, 1995.

[7]

F. T. Durso and A. R. Dattel, "SPAM: The real-time assessment of SA," in A cognitive approach to situation awareness: Theory and application, S. Banbury and S. Tremblay, Eds., 2004, pp. 137--154.

[8]

A. D. Andre, C. D. Wickens, and L. Moorman, "Display formatting techniques for improving situation awareness in the aircraft cockpit," The International Journal of Aviation Psychology, pp. 205--218, 1991.

[9]

D. G. Jones and M. R. Endsley, "Sources of situation awareness errors in aviation.," Aviation, Space and Environmental Medicine, vol. 67, pp. 507--512, 1996.

[10]

M. R. Endsley and R. P. Smith, "Attention distribution and decision making in tactical air combat," Human Factors, vol. 38, pp. 232--249, 1996.

[11]

M. R. Endsley and M. D. Rodgers, "Distribution of attention, situation awareness, and workload in a passive air traffic control task: Implications for operational errorsand automation," Air Traffic Control Quarterly, vol. 6, pp. 21--44, 1998.

[12]

L. Gugerty, "Evidence from a partial report task for forgetting in dynamic spatial memory," Human Factors, vol. 40, pp. 498--508, 1998.

[13]

L. Gugerty, M. Rakauskas, and J. Brooks, "Effects of remote and in-person verbal interactions on verbalization rates and attention to dynamic spatial scenes," Accident analysis and prevention, vol. 36, pp. 1029--1043, 2004.

[14]

L. Gugerty, "Situation awareness in driving," in Handbook for Driving Simulation in Engineering Medicine and Psychology, J. Lee, M. Rizzo, D. Fisher, and J. Caird, Eds.: CRC Press, in press.

[15]

W. J. Horrey, C. D. Wickens, and K. P. Consalus, "Modeling drivers' visual attention allocation while interacting with in-vehicle technologies," Journal of Experimental Psychology: Applied, vol. 12, pp. 67--78, 2006.

[16]

G. O. Einstein, M. A. McDaniel, and M. Brandimonte, Prospective memory: Theory and application: L. Erlbaum, 1996.

[17]

J. G. Trafton, E. M. Altmann, D. P. Brock, and F. E. Mintz, "Preparing to resume an interrupted task: Effects of prospective goal encoding and retrospective rehearsal," International Journal of Human-Computer Studies, vol. 58, pp. 583--603, 2003.

Digital Library

[18]

G. J. Zelinsky and L. C. Loschky, "Using eye movements to study working memory rehearsal for objects in visual scenes," in Proceedings of the 31st Annual Conference of the Cognitive Science Society, N. Taatgen, H. V. Rijn, and L. Schomaker, Eds., 2009.

[19]

C. D. Wickens, J. Goh, J. Helleberg, and W. J. Horrey, "Attentional models of multitask pilot performance using advnaced display technology," Human Factors, vol. 45, pp. 360--380, 2003.

[20]

L. C. Thomas and C. D. Wickens, "Eye-tracking and individual differences in off-normal event detection when flying with a synthetic vision system display," in Proceedings of the 48th Annual Meeting of the Human Factors Society, 2004.

[21]

L. Gugerty, "Situation awareness during driving: Explicit and implicit knowledge in dynamic spatial memory," Journal of Experimental Psychology:Applied, vol. 3, pp. 42--66, 1997.

[22]

P. Salmon, N. A. Stanton, G. Walker, and D. Green, "Situation measurement: A review of applicability for C4i environments," Applied Ergonomics, vol. 37, pp. 225--238, 2006.

[23]

K. Rayner, "Eye movements in reading and information processing: 20 years of research," Psychological Bulletin, vol. 85, pp. 618--660, 1998.

[24]

K. Rayner and R. K. Morris, "Do eye-movements reflect higher order processes in reading?," in From eye to mind. Information aquisition in perception, search, and reading, R. Groner, D. d'Ydewalle, and R. Parham, Eds. Amsterdam: North-Holland, 1990, pp. 191--204.

[25]

M. A. Just and P. A. Carpenter, "Eye fixations and cognitive processes," Cognitive Psychology, vol. 8, pp. 441--480, 1976.

[26]

G. F. Wilson and C. A. Russell, "Performance enhancement in an unihabited air vehicle task using psychophysiologically determined adpative aiding," Human Factors, vol. 49, pp. 1005--1018, 2007.

[27]

G. F. Wilson and C. A. Russell, "Operator functional state classification using multiple psychophysiological features in an air traffic control task," Human Factors, vol. 45, pp. 381--389, 2003.

[28]

C.-Y. J. Peng, K. L. Lee, and G. M. Ingersoll, "An introduction to logistic regression analysis and reporting," The Journal of Educational Research, vol. 96, pp. 3--14, 2002.

[29]

Y. Boussemart and M. L. Cummings, "Behavioral recognition and prediction of an operator supervising multiple heterogeneous unmanned vehicles," in Humans operating unmanned systems, 2008.

[30]

C. Nehme, "Modeling human supervisory control in heterogeneous unmanned vehicle systems," in Deptartment of Aeronautics and Stronautics. Cambridge, MA: MIT, 2009.

[31]

T. Fawcett, "An introduction to ROC analysis," Pattern Recognition Letters, vol. 27, pp. 861--874, 2006.

Digital Library

[32]

N. A. Macmillan and C. D. Creelman, Detection Theorys: A User's Guide: Lawrence Erlbaum Associates, 2005.

[33]

L. Itti and C. Koch, "A saliency-based search mechanism for overt and covert shifts of visual attention," Vision Research, vol. 40, pp. 1489--1506, 2000.

[34]

R. M. Ratwani, J. M. McCurry, and J. G. Trafton, "Predicting postcompletion errors using eye movements," in Proceedings of the 2008 Conference on Human Factors in Computing Systems, CHI 2008, M. Czerwinski, A. M. Lund, and D. S. Tan, Eds.: ACM, 2008, pp. 539--542.

Digital Library

[35]

R. M. Ratwani and J. G. Trafton, "Developing a Predictive Model of Postcompletion Errors," in Proceedings of the 31st Annual Meeting of the Cognitive Science Society, 2009.

[36]

M. R. Endsley and E. O. Kiris, "The out-of-the-loop performance problem and level of control in automation," Human Factors, vol. 37, 1995.

[37]

R. Parasuraman, T. B. Sheridan, and C. D. Wickens, "A model for types and levels of human interaction with automation," IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and human, vol. 30, pp. 286--297, 2000.

Digital Library

Cited By

Watson GPapelis YHicks KMittal SRisco Martin JElfrey PCetinkaya DSeveringhaus R(2016)Simulation-based environment for the eye-tracking control of tele-operated mobile robotsProceedings of the Modeling and Simulation of Complexity in Intelligent, Adaptive and Autonomous Systems 2016 (MSCIAAS 2016) and Space Simulation for Planetary Space Exploration (SPACE 2016)10.5555/2962664.2962668(1-7)Online publication date: 3-Apr-2016
https://dl.acm.org/doi/10.5555/2962664.2962668
Li WBraithwaite GGreaves MHsu CLin SBoy G(2016)The evaluation of military pilot's attention distributions on the flight deckProceedings of the International Conference on Human-Computer Interaction in Aerospace10.1145/2950112.2964588(1-6)Online publication date: 14-Sep-2016
https://dl.acm.org/doi/10.1145/2950112.2964588
Li WBraithwaite GYu C(2014)The Investigation of Pilots' Eye Scan Patterns on the Flight Deck during an Air-to-Surface Task11th International Conference on Engineering Psychology and Cognitive Ergonomics - Volume 853210.1007/978-3-319-07515-0_33(325-334)Online publication date: 22-Jun-2014
https://dl.acm.org/doi/10.1007/978-3-319-07515-0_33
Show More Cited By

Index Terms

Single operator, multiple robots: an eye movement based theoretic model of operator situation awareness
1. Applied computing
  1. Law, social and behavioral sciences
2. General and reference
  1. Document types

Recommendations

Measuring Situation Awareness in Control Room Teams
Engineering Psychology and Cognitive Ergonomics. Cognition and Design
Abstract
The importance of monitoring activities in control rooms continues to increase. Teams of operators are required to monitor a system for any abnormal system behavior, and must be able to exert manual control over the system in case of automation ...
Towards Real-time Probabilistic Evaluation of Situation Awareness from Human Gaze in Human-Robot Interaction
HRI '17: Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction

Human attention processes play a major role for optimization in human-robot interaction (HRI). This work describes a novel methodology to measure situation awareness in real-time from gaze interaction with scene objects of interest using eye tracking ...
Understanding situation awareness in SOCs, a systematic literature review
Abstract
Situation awareness is shown through human factors research to be a valuable construct to understand and improve how humans perform while operating complex systems in critical environments. Within cyber security one such environment is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '10: Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction

March 2010

400 pages

ISBN:9781424448937

General Chairs:
Pamela Hinds
Stanford University, USA
,
Hiroshi Ishiguro
Osaka University, Japan
,
Program Chairs:
Takayuki Kanda
ATR, Japan
,
Peter Kahn
University of Washington, USA

Sponsors

Publisher

IEEE Press

Publication History

Published: 02 March 2010

Check for updates

Author Tags

Qualifiers

Research-article

Conference

HRI 10

Sponsor:

HRI 10: International Conference on Human Robot Interaction

March 2 - 5, 2010

Osaka, Japan

Acceptance Rates

HRI '10 Paper Acceptance Rate 26 of 124 submissions, 21%;

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
362
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Watson GPapelis YHicks KMittal SRisco Martin JElfrey PCetinkaya DSeveringhaus R(2016)Simulation-based environment for the eye-tracking control of tele-operated mobile robotsProceedings of the Modeling and Simulation of Complexity in Intelligent, Adaptive and Autonomous Systems 2016 (MSCIAAS 2016) and Space Simulation for Planetary Space Exploration (SPACE 2016)10.5555/2962664.2962668(1-7)Online publication date: 3-Apr-2016
https://dl.acm.org/doi/10.5555/2962664.2962668
Li WBraithwaite GGreaves MHsu CLin SBoy G(2016)The evaluation of military pilot's attention distributions on the flight deckProceedings of the International Conference on Human-Computer Interaction in Aerospace10.1145/2950112.2964588(1-6)Online publication date: 14-Sep-2016
https://dl.acm.org/doi/10.1145/2950112.2964588
Li WBraithwaite GYu C(2014)The Investigation of Pilots' Eye Scan Patterns on the Flight Deck during an Air-to-Surface Task11th International Conference on Engineering Psychology and Cognitive Ergonomics - Volume 853210.1007/978-3-319-07515-0_33(325-334)Online publication date: 22-Jun-2014
https://dl.acm.org/doi/10.1007/978-3-319-07515-0_33
Frische FLüdtke AMackay WBrewster SBødker S(2013)Supervisory guide part ICHI '13 Extended Abstracts on Human Factors in Computing Systems10.1145/2468356.2468377(109-114)Online publication date: 27-Apr-2013
https://dl.acm.org/doi/10.1145/2468356.2468377
Vrzakova HBednarik R(2012)Hard lessons learnedProceedings of the 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction10.1145/2401836.2401843(1-6)Online publication date: 26-Oct-2012
https://dl.acm.org/doi/10.1145/2401836.2401843

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents