research-article

Let Me Finish before I Take Over: Towards Attention Aware Device Integration in Highly Automated Vehicles

Authors:

Philipp Wintersberger,

Andreas Riener,

Clemens Schartmüller,

Anna-Katharina Frison,

Klemens WeiglAuthors Info & Claims

AutomotiveUI '18: Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

Pages 53 - 65

https://doi.org/10.1145/3239060.3239085

Published: 23 September 2018 Publication History

Abstract

A major promise of automated vehicles is to render it possible for drivers to engage in nondriving related tasks, a setting where the execution pattern will switch from concurrent to sequential multitasking. To allow drivers to safely and efficiently switch between multiple activities (including vehicle control in case of Take-Over situations), we postulate that future vehicles should incorporate capabilities of attentive user interfaces, that precisely plan the timing of interruptions based on driver availability. We propose an attention aware system that issues Take-Over Requests (1) at emerging task boundaries and (2) directly on consumer devices such as smartphones or tablets. Results of a driving simulator study (N=18), where we evaluated objective, physiological, and subjective measurements, confirm our assumption: attention aware Take-Over Requests have the potential to reduce stress, increase Take-Over performance, and can further raise user acceptance/trust. Consequently, we emphasize to implement attentive user interfaces in future vehicles.

References

[1]

Piotr D. Adamczyk and Brian P. Bailey. 2004. If Not Now, when?: The Effects of Interruption at Different Moments Within Task Execution. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). ACM, New York, NY, USA, 271--278.

Digital Library

[2]

A. S. Arif and W. Stuerzlinger. 2009. Analysis of text entry performance metrics. In 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH). 100--105.

[3]

Brian P. Bailey and Joseph A. Konstan. 2006. On the need for attention-aware systems: Measuring effects of interruption on task performance, error rate, and affective state. Computers in Human Behavior 22, 4 (2006), 685 -- 708. Attention aware systems.

[4]

Mathias Benedek and Christian Kaernbach. 2010. A continuous measure of phasic electrodermal activity. Journal of Neuroscience Methods 190, 1 (2010), 80--91.

[5]

Mathias Benedek and Christian Kaernbach. 2018. Ledalab Software. (2018). http://www.ledalab.de/

[6]

W. Boucsein. 2004. Electrodermal measurement. In Handbook of Ergonomics and Human Factors Methods, Neville Stanton, Alan Hedge, H.W. Hendrick, K.A. Brookhuis, and E. Salas (Eds.). Taylor & Francis, London.

[7]

Wolfram Boucsein, Don C. Fowles, Sverre Grimnes, Gershon Ben-Shakhar, Walton T. Roth, Michael E. Dawson, and Diane L. Filion. 2012. Publication recommendations for electrodermal measurements. Psychophysiology 49, 8 (2012), 1017--1034.

[8]

JJ Braithwaite, DG Watson, Jones Robert, and Rowe Mickey. 2013. A Guide for Analysing Electrodermal Activity (EDA) & Skin Conductance Responses (SCRs) for Psychological Experiments. ... (2013), 1--42.

[9]

K. A. Brookhuis, D. DE Waard, and S. H. Fairclough. 2003. Criteria for driver impairment. Ergonomics 46, 5 (apr 2003), 433--445.

[10]

James D Brown and Warren J Huffman. 1972. Psychophysiological measures of drivers under actual driving conditions. Journal of Safety Research 4, 4 (1972), 172--178.

[11]

Andreas Bulling. 2016. Pervasive Attentive User Interfaces. IEEE Computer 49, 1 (2016), 94--98.

Digital Library

[12]

C Collet, A Clarion, M Morel, A Chapon, and C Petit. 2009. Physiological and behavioural changes associated to the management of secondary tasks while driving. Applied Ergonomics 40, 6 (2009), 1041--1046.

[13]

SAE On-Road Automated Vehicle Standards Committee and others. 2014. Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems. SAE International (2014).

[14]

SAE On-Road Automated Vehicle Standards Committee and others. 2016. Human Factors Definitions for Automated Driving and Related Research Topics. SAE International (2016).

[15]

Fred D Davis. 1985. A technology acceptance model for empirically testing new end-user information systems: Theory and results. Ph.D. Dissertation. Massachusetts Institute of Technology.

[16]

F. DeRemer and H. H. Kron. 1976. Programming-in-the-Large Versus Programming-in-the-Small. IEEE Transactions on Software Engineering SE-2, 2 (June 1976), 80--86.

Digital Library

[17]

Stenographisches Protokoll des Nationalrates XXV. 2016. 32. KFG-Novelle. Bundesgesetz Oesterreich. (15 March 2016). Retrieved April 22, 2018 from https://goo.gl/HedM9u.

[18]

Stefan Diewald, Andreas Möller, Luis Roalter, Matthias Kranz, and others. 2011. Mobile device integration and interaction in the automotive domain. In AutoNUI: automotive natural user interfaces workshop at the 3rd international conference on automotive user interfaces and interactive vehicular applications (AutomotiveUI 2011).

[19]

Mica R. Endsley and Esin O. Kiris. 1995. The Out-of-the-Loop Performance Problem and Level of Control in Automation. Human Factors 37, 2 (1995), 381--394.

[20]

Alexander Eriksson and Neville A. Stanton. 2017. Takeover Time in Highly Automated Vehicles: Noncritical Transitions to and From Manual Control. Human Factors 59, 4 (2017), 689--705. 28124573.

[21]

Anna-Katharina Frison, Philipp Wintersberger, Andreas Riener, and Clemens Schartmüller. 2017. Driving Hotzenplotz: A Hybrid Interface for Vehicle Control Aiming to Maximize Pleasure in Highway Driving. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '17). ACM, New York, NY, USA, 236--244.

Digital Library

[22]

Christian Gold, Daniel Damböck, Lutz Lorenz, and Klaus Bengler. 2013. Take over! How long does it take to get the driver back into the loop? Proceedings of the Human Factors and Ergonomics Society Annual Meeting 57, 1 (2013), 1938--1942.

[23]

Christian Gold, Moritz KÃűrber, David Lechner, and Klaus Bengler. 2016. Taking Over Control From Highly Automated Vehicles in Complex Traffic Situations: The Role of Traffic Density. Human Factors 58, 4 (2016), 642--652. 26984515.

[24]

Paul Green. 2004. Driver Distraction, Telematics Design, and Workload Managers: Safety Issues and Solutions. In Convergence International Congress & Exposition On Transportation Electronics. Convergence Transportation Electronics Association.

[25]

S. G. Hart. 2006. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50, 9 (oct 2006), 904--908.

[26]

Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Human Mental Workload, Peter A. Hancock and Najmedin Meshkati (Eds.). Advances in Psychology, Vol. 52. North-Holland, 139 -- 183.

[27]

Eric Horvitz. 1999. Principles of Mixed-initiative User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '99). ACM, New York, NY, USA, 159--166.

Digital Library

[28]

Shamsi T. Iqbal and Brian P. Bailey. 2005. Investigating the Effectiveness of Mental Workload As a Predictor of Opportune Moments for Interruption. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). ACM, New York, NY, USA, 1489--1492.

Digital Library

[29]

Jiun-Yin Jian, Ann M. Bisantz, and Colin G. Drury. 2000. Foundations for an Empirically Determined Scale of Trust in Automated Systems. International Journal of Cognitive Ergonomics 4, 1 (2000), 53--71.

[30]

Wolfgang Fastenmeier Joerg Kubitzki. 2016. Ablenkung durch moderne Informations- und Kommunikationstechniken und soziale Interaktion bei Autofahrern. Allianz AG.

[31]

SeungJun Kim, Jaemin Chun, and Anind K. Dey. 2015. Sensors Know When to Interrupt You in the Car: Detecting Driver Interruptibility Through Monitoring of Peripheral Interactions. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 487--496.

Digital Library

[32]

Gwendolin Knappe, Andreas Keinath, Klaus Bengler, and Cristina Meinecke. 2007. Driving simulator as an evaluation tool--assessment of the influence of field of view and secondary tasks on lane keeping and steering performance. In 20th International Technical Conference on the Enhanced Safety of Vehicles (ESV) National Highway Traffic Safety Administration.

[33]

Tao Lin, Masaki Omata, Wanhua Hu, and Atsumi Imamiya. 2005. Do physiological data relate to traditional usability indexes? Proceedings of the 17th Australia conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future (2005), 1--10. http://portal.acm.org/citation.cfm?id=1108368.1108405

Digital Library

[34]

Christoph Linz. 2017. Handy am Steuer wird teurer. Stuttgarter Nachrichten. (21 September 2017). Retrieved August 22, 2017 from https://goo.gl/FopUCo.

[35]

Stefan Mattes. 2003. The lane-change-task as a tool for driver distraction evaluation. Quality of Work and Products in Enterprises of the Future 57 (2003).

[36]

Alexander G. Mirnig, Magdalena Gärtner, Arno Laminger, Alexander Meschtscherjakov, Sandra Trösterer, Manfred Tscheligi, Rod McCall, and Fintan McGee. 2017. Control Transition Interfaces in Semiautonomous Vehicles: A Categorization Framework and Literature Analysis. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '17). ACM, New York, NY, USA, 209--220.

Digital Library

[37]

Frederik Naujoks, Christoph Mai, and Alexandra Neukum. 2014. The effect of urgency of takeover requests during highly automated driving under distraction conditions. Advances in Human Aspects of Transportation 7, Part I (2014), 431.

[38]

Frederik Naujoks, Katharina Wiedemann, and Nadja Schömig. 2017. The Importance of Interruption Management for Usefulness and Acceptance of Automated Driving. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '17). ACM, New York, NY, USA, 254--263.

Digital Library

[39]

Elliot Njus. 2017. Oregon's new distracted driving law takes effect Oct. 1: what you need to know. The Oregonian. (20 September 2017). Retrieved August 22, 2018 from https://goo.gl/W9bCKK.

[40]

Bastian Pfleging, Maurice Rang, and Nora Broy. 2016. Investigating User Needs for Nondriving-related Activities During Automated Driving. In Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia (MUM '16). ACM, New York, NY, USA, 91--99.

Digital Library

[41]

Ioannis Politis, Stephen Brewster, and Frank Pollick. 2015. Language-based Multimodal Displays for the Handover of Control in Autonomous Cars. In Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '15). ACM, New York, NY, USA, 3--10.

Digital Library

[42]

Jonas Radlmayr, Christian Gold, Lutz Lorenz, Mehdi Farid, and Klaus Bengler. 2014. How Traffic Situations and Non-Driving Related Tasks Affect the Take-Over Quality in Highly Automated Driving. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 58, 1 (2014), 2063--2067.

[43]

Shadan Sadeghian Borojeni, Susanne C.J. Boll, Wilko Heuten, Heinrich H. Bülthoff, and Lewis Chuang. 2018. Feel the Movement: Real Motion Influences Responses to Takeover Requests in Highly Automated Vehicles. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 246, 13 pages.

Digital Library

[44]

Nadine B. Sarter and David D. Woods. 1995. How in the World Did We Ever Get into That Mode? Mode Error and Awareness in Supervisory Control. Human Factors 37, 1 (1995), 5--19.

[45]

Mahadev Satyanarayanan. 2001. Pervasive computing: Vision and challenges. IEEE Personal communications 8, 4 (2001), 10--17.

[46]

R. William Soukoreff and I. Scott MacKenzie. 2001. Measuring Errors in Text Entry Tasks: An Application of the Levenshtein String Distance Statistic. In CHI '01 Extended Abstracts on Human Factors in Computing Systems (CHI EA '01). ACM, New York, NY, USA, 319--320.

Digital Library

[47]

R. William Soukoreff and I. Scott MacKenzie. 2003. Metrics for Text Entry Research: An Evaluation of MSD and KSPC, and a New Unified Error Metric. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, New York, NY, USA, 113--120.

Digital Library

[48]

Serge Thill, Maria Riveiro, and Maria Nilsson. 2015. Perceived intelligence as a factor in (semi-) autonomous vehicle UX. In " Experiencing Autonomous Vehicles: Crossing the Boundaries between a Drive and a Ride" workshop in conjunction with CHI2015.

[49]

Tomer Toledo and David Zohar. 2007. Modeling Duration of Lane Changes. Transportation Research Record 1999, 1 (2007), 71--78.

[50]

Viswanath Venkatesh and Fred D. Davis. 2000. A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies. Management Science 46, 2 (2000), 186--204.

Digital Library

[51]

Roel Vertegaal and others. 2003. Attentive user interfaces. Commun. ACM 46, 3 (2003), 30--33.

Digital Library

[52]

Gabriela Villalobos-Zúñiga, Tuomo Kujala, and Antti Oulasvirta. 2016. T9+HUD: Physical Keypad and HUD Can Improve Driving Performance While Typing and Driving. In Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Automotive'UI 16). ACM, New York, NY, USA, 177--184.

Digital Library

[53]

David Watson, Lee Anna Clark, and Auke Tellegen. 1988. Development and validation of brief measures of positive and negative affect: the PANAS scales. Journal of personality and social psychology 54, 6 (1988), 1063.

[54]

Philipp Wintersberger, Paul Green, and Andreas Riener. 2017. Am I Driving or Are You or Are We Both? A Taxonomy for Handover and Handback in Automated Driving. (2017).

[55]

Philipp Wintersberger and Andreas Riener. 2016. Trust in technology as a safety aspect in highly automated driving. i-com 15, 3 (2016), 297--310.

Cited By

Krstačić RŽužić AOrehovački T(2024)Safety Aspects of In-Vehicle Infotainment Systems: A Systematic Literature Review from 2012 to 2023Electronics10.3390/electronics1313256313:13(2563)Online publication date: 29-Jun-2024
https://doi.org/10.3390/electronics13132563
Srour Zreik RGoodge THarvey MBrewster S(2024)Where's my TOR?: Evaluating the Effect of Take-Over Request Source on Older Drivers' Control Transition in Level 3 CarsProceedings of the ACM on Human-Computer Interaction10.1145/36765338:MHCI(1-25)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676533
Ch NFortier JJanssen CShaer OMills CKun A(2024)Text a Bit Longer or Drive Now? Resuming Driving after Texting in Conditionally Automated CarsProceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3640792.3675737(13-22)Online publication date: 22-Sep-2024
https://dl.acm.org/doi/10.1145/3640792.3675737
Show More Cited By

Index Terms

Let Me Finish before I Take Over: Towards Attention Aware Device Integration in Highly Automated Vehicles
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. HCI theory, concepts and models
  2. Ubiquitous and mobile computing
    1. Empirical studies in ubiquitous and mobile computing

Recommendations

Assisting Drivers with Ambient Take-Over Requests in Highly Automated Driving
Automotive'UI 16: Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

Take-over situations in highly automated driving occur when drivers have to take over vehicle control due to automation shortcomings. Due to high visual processing demand of the driving task and time limitation of a take-over maneuver, appropriate user ...
From reading to driving: priming mobile users for take-over situations in highly automated driving
MobileHCI '18: Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services

Highly automated vehicles, occasionally require users to resume vehicle control from non-driving related tasks by issuing cues called take-over request (TOR). Due to being engaged in non-driving related tasks (NDRT), users have a decreased level of ...
Steer-By-WiFi: Lateral Vehicle Control for Take-Overs with Nomadic Devices
AutomotiveUI '18: Adjunct Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

Automated vehicles could omit traditional steering controls to provide larger spaces for driver-passengers or prevent unnecessary interventions. However, manual control could still be necessary to provide manual driving fun or respond to Take-Over ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

AutomotiveUI '18: Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

September 2018

374 pages

ISBN:9781450359467

DOI:10.1145/3239060

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 September 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

German Federal Ministry of Education and Research (BMBF)

Conference

AutomotiveUI '18

Sponsor:

SIGCHI

AutomotiveUI '18: 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

September 23 - 25, 2018

ON, Toronto, Canada

Acceptance Rates

Overall Acceptance Rate 248 of 566 submissions, 44%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

57
Total Citations
View Citations
1,201
Total Downloads

Downloads (Last 12 months)74
Downloads (Last 6 weeks)9

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Krstačić RŽužić AOrehovački T(2024)Safety Aspects of In-Vehicle Infotainment Systems: A Systematic Literature Review from 2012 to 2023Electronics10.3390/electronics1313256313:13(2563)Online publication date: 29-Jun-2024
https://doi.org/10.3390/electronics13132563
Srour Zreik RGoodge THarvey MBrewster S(2024)Where's my TOR?: Evaluating the Effect of Take-Over Request Source on Older Drivers' Control Transition in Level 3 CarsProceedings of the ACM on Human-Computer Interaction10.1145/36765338:MHCI(1-25)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676533
Ch NFortier JJanssen CShaer OMills CKun A(2024)Text a Bit Longer or Drive Now? Resuming Driving after Texting in Conditionally Automated CarsProceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3640792.3675737(13-22)Online publication date: 22-Sep-2024
https://dl.acm.org/doi/10.1145/3640792.3675737
Ebel PBazilinskyy PColley MGoodridge CHock PJanssen CSandhaus HSrinivasan AWintersberger P(2024)Changing Lanes Toward Open Science: Openness and Transparency in Automotive User ResearchProceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3640792.3675730(94-105)Online publication date: 22-Sep-2024
https://dl.acm.org/doi/10.1145/3640792.3675730
Goodge TPollick FBrewster S(2024)Can You Hazard a Guess?: Evaluating the Effect of Augmented Reality Cues on Driver Hazard PredictionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642300(1-28)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642300
Lingler ATalypova DJokinen JOulasvirta AWintersberger P(2024)Supporting Task Switching with Reinforcement LearningProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642063(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642063
Janssen CPraetorius LBorst J(2024)PREDICTOR: A tool to predict the timing of the take-over response process in semi-automated drivingTransportation Research Interdisciplinary Perspectives10.1016/j.trip.2024.10119227(101192)Online publication date: Sep-2024
https://doi.org/10.1016/j.trip.2024.101192
Deng MGluck AZhao YLi DMenassa CKamat VBrinkley J(2024)An analysis of physiological responses as indicators of driver takeover readiness in conditionally automated drivingAccident Analysis & Prevention10.1016/j.aap.2023.107372195(107372)Online publication date: Feb-2024
https://doi.org/10.1016/j.aap.2023.107372
Lingler ATalypova DDraxler FSchneegass CDingler TWintersberger P(2023)MuM'23 Workshop on Interruptions and Attention ManagementProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3626706(548-551)Online publication date: 3-Dec-2023
https://dl.acm.org/doi/10.1145/3626705.3626706
Ebel PBazilinskyy PHwang AJu WSandhaus HSrinivasan AYang QWintersberger P(2023)Breaking Barriers: Workshop on Open Data Practices in AutoUI ResearchAdjunct Proceedings of the 15th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3581961.3609835(227-230)Online publication date: 18-Sep-2023
https://dl.acm.org/doi/10.1145/3581961.3609835
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten