Abstract
The objective of this paper is to present a new adaptive automation concept which offers an innovative ‘team’ centred approach to solving human factors/workload management problems. The A-PiMod concept/approach is defined by the concept of partnership – specifically, the “Third Pilot” and the crew and automation are in charge together. We are proposing partnership as opposed to dynamic changes in control function where changes can be controlled autonomously by the system. In support of this, a new multimodal concept is proposed which supports improved assessment of crew state/workload (i.e. information inputs re crew activity/interactions provides a means to communicate with the crew in relation to crew state and decision support, and allows for flexible crew/cockpit interaction).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Kaber, D.B., Prinzel, L.J.: Adaptive and Adaptable Automation Design: A Critical Review of the Literature and Recommendations for Future Research. NASA/TM-2006-214504 (2006)
Hilburn, B.J., Byrne, E., Parasuraman, R.: The effect of adaptive air traffic control (ATC) decision aiding on controller mental workload. In: Mouloua, M., Koonce, J.M. (eds.) Human–Automation Interaction: Research and Practice, pp. 84–91. Lawrence Erlbaum Associates Inc, Mahwah, NJ (1997)
Kaber, D.B., Riley, J.M.: Adaptive automation of a dynamic control task based on secondary task workload measurement. Int. J. Cogn. Ergon. 3, 169–187 (2006)
Flight AF 447 Final Report on the accident on 1st June 2009 to the Airbus A330-203 registered F-GZCP operated by Air France flight AF 447 Rio de Janeiro (Published July 2012). Retrieved from Bureau d’Enquêtes et d’Analyses pour la sécurité de l’aviation civile (BEA), 1 June 2009. http://www.bea.aero/docspa/2009/f-cp090601.en/pdf/f-cp090601.en.pdf
Flight Spainair 5022. Final Report on the accident on 20th August 2008 involving a McDonnell Douglas DC-9-82 (MD-82) registration EC-HFP operated by Spainair at Madrid-Barajas Airport (Published 8 October 2008). Retrieved from Comisión Investigatión de Accidentes e Incidentes de Aviación Civil (CIAIAC), 20 August 2008. http://www.fomento.es/NR/rdonlyres/EC47A855-B098-409E-B4C8-9A6DD0D0969F/107087/2008_032_A_ENG.pdf
Flight Helios Airways HCY522. Final Report on the accident on 14th August 2005 involving a Boeing 737-31S registration 5B-DBY operated by Helios Airways at Grammatiko, Hellas (Published November 2006). Retrieved from Air Accident Investigation & Aviation Safety Board (AAIASB). 14 August 2005. http://www.moi.gov.cy/moi/pio/pio.nsf/All/F15FBD7320037284C2257204002B6243/$file/FINAL%20REPORT%205B-DBY.pdf
Flight China Airlines 140. Final Report on the accident on 26th April 1994 involving an Airbus Industrie A300B4-662R registration B1816 operated by China Airlines at Nagoya Airport (Published 19 July 1996). Retrieved from Aircraft Accident Investigation Commission. 26 April 1994. http://www.skybrary.aero/bookshelf/books/808.pdf
Flight Air Inter 148. Final Report on the accident on 20th January 1992 involving an Airbus A320 registration F-GGED operated by Air Inter Airlines in Vosges Mountains (near Mont Sainte-Odile). Retrieved from Bureau d’Enquêtes et d’Analyses pour la sécurité de l’aviation civile (BEA), 20 January 1992. http://www.bea.aero/docspa/1992/f-ed920120/htm/f-ed920120.html
Cahill, J., Callari, T.C.: A novel human machine interaction (HMI) design/evaluation approach supporting the advancement of improved automation concepts to enhance flight safety. In: de Waard, D., Sauer, J., Röttger, S., Kluge, A., Manzey, D., Weikert, C., Toffetti, A., Wiczorek, R., Brookhuis, K., Hoonhout, H. (Eds.). In: Proceeding of the Ergonomics Society Europe Chapter 2014 Annual Conference “Human Factors in high reliability industries”, Lisbon, Portugal. (2015a). http://www.hfes-europe.org/human-factors-high-reliability-industries-2/
Cahill, J., Callari, T.: Stakeholder involvement in evaluation: lessons learned in the A-PiMod Project. Presentation at the Annual Meeting of the Irish Ergonomics Society. Dublin, Ireland, May 2015 (2015b)
Cahill, J, Callari, T. Javaux, D., Fortmann, F., Hasselberg, A. A-PiMod: a new approach to solving human factors problems with automation Paper presented at the HCI 2016 International Conference, Toronto Canada, July 2016
Stroeve, S., Van Doorn, B.A., Cahill, J. A Safety impact quantification approach for early stage innovative aviation concepts: Application to a third pilot adaptive automation concept. Paper presented at SESAR Innovation Days, Delft, November 2016
Wenger, E., McDermott, R.A., Snyder, W.: Cultivating Communities of Practice: A Guide to Managing Knowledge. Harvard Business Press, Boston (2002)
Cousins, J.B., Whitmore, E., Shulha, L.: Arguments for a common set of principles for collaborative inquiry in evaluation. Am. J. Eval. 34(1), 7–22 (2013)
Javaux, D., Fortmann, F., Möhlenbrink, C.: Adaptive human-automation cooperation: a general architecture for the cockpit and its application in the A-PiMod project. In: Proceedings of the 7th International Conference on Advanced Cognitive Technologies and Applications (COGNITIVE 2015). International Academy, Research, and Industry Association (IARIA) (2015). ISBN: 978-1-61208-390-2
Fortmann, F., Cahill, J, Callari, T. Javaux, D., Hasselberg, A. (2016). Developing a feedback system to augment pilots monitoring performance. Paper presented at 2016 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), San Diego, US, 1–25 March 2016
Acknowledgments
The research leading to these results/preliminary outcomes has received funding from the European Commission’s Seventh Framework Programme (FP7/2007-2013) under grant agreement N. 605141 - Applying Pilot Models for Safety Aircraft (A-PiMod) Project. We would like to thank member of the A-PiMod Project Team and our COP members – particularly, Paul Cullen, William Butler, Martin Duffy and Stephen Duffy.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
Appendices
1.1 Systems to Monitor Observed Behavior
Eye Tracking.
In both sets of simulator evaluation sessions, SMI Eye Tracking Glasses 2 were used to measure the gaze positions. Please note that his was demonstrated – but not implemented in real time. SMI Eye Tracking Glasses is a binocular tracking device which operates with 60 Hz. It is connected with a Laptop via USB on which videos of the eyes and the scene camera are recorded and the gaze position is calculated. The system is combined with an A.R.T Optical Head Tracking system. Retro reflective targets are attached to the glasses which are recorded by infrared cameras installed in the simulator. This system allows us to calculate the head position and orientation. With both systems combined, it is possible to calculate a 3D gaze vector for each eye.
Gesture Recognition.
The Gesture recognition is meant to recognize the upper body parts of a human operator – aeronautic pilot – in order to detect “implicit gestures”. Implicit gestures refer to movements of the upper body parts, which are normal actions taken by the crew (e.g. controlling different parts of the cockpit, or interaction among the crew).
Head Pose Recognition.
The head pose estimation functionality serves to provide information about where the pilot is looking within the cockpit (which instruments, screens, control elements, etc.). The technology is designed to be completely passive and non-intrusive in the sense that the pilot does not wear (or otherwise consciously interact with) any additional pieces of equipment, such as eye-tracking glasses. Also, the head pose estimation device does not emit any infrared light, which would be the case for contemporary remote eye trackers or depth cameras (based on structured light projection or infrared time-of-flight sensors). In A-PiMod, the first application of the said technology in the cockpit is detecting “missed events” – when the pilot is provided with a piece of information by the cockpit, but she misses it by not looking at the appropriate display for a time period. The cockpit display (MC-M Display) provides a notification of the missed event. Depending on the Pilots response, the saliency of such message is increased. The second application is contribution to the estimation of the pilot’s state of mind and workload level from the patterns of the head motion.
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Cahill, J. et al. (2017). Adaptive Automation and the Third Pilot: Managing Teamwork and Workload in an Airline Cockpit. In: Longo, L., Leva, M. (eds) Human Mental Workload: Models and Applications. H-WORKLOAD 2017. Communications in Computer and Information Science, vol 726. Springer, Cham. https://doi.org/10.1007/978-3-319-61061-0_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-61061-0_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-61060-3
Online ISBN: 978-3-319-61061-0
eBook Packages: Computer ScienceComputer Science (R0)