research-article

Cognitive rhythms: unobtrusive and continuous sensing of alertness using a mobile phone

Authors:

Saeed Abdullah,

Elizabeth L. Murnane,

Julie A. Kientz,

Tanzeem ChoudhuryAuthors Info & Claims

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

Pages 178 - 189

https://doi.org/10.1145/2971648.2971712

Published: 12 September 2016 Publication History

Abstract

Throughout the day, our alertness levels change and our cognitive performance fluctuates. The creation of technology that can adapt to such variations requires reliable measurement with ecological validity. Our study is the first to collect alertness data in the wild using the clinically validated Psychomotor Vigilance Test. With 20 participants over 40 days, we find that alertness can oscillate approximately 30% depending on time and body clock type and that Daylight Savings Time, hours slept, and stimulant intake can influence alertness as well. Based on these findings, we develop novel methods for unobtrusively and continuously assessing alertness. In estimating response time, our model achieves a root-mean-square error of 80.64 milliseconds, which is significantly lower than the 500ms threshold used as a standard indicator of impaired cognitive ability. Finally, we discuss how such real-time detection of alertness is a key first step towards developing systems that are sensitive to our biological variations.

References

[1]

Khaled Abdel-Kader, Manisha Jhamb, Lee Anne Mandich, Jonathan Yabes, Robert M Keene, Scott Beach, Daniel J Buysse, and Mark L Unruh. 2014. Ecological momentary assessment of fatigue, sleepiness, and exhaustion in ESKD. BMC nephrology 15 (2014), 1.

[2]

Saeed Abdullah, Mark Matthews, Elizabeth L Murnane, Geri Gay, and Tanzeem Choudhury. 2014. Towards circadian computing: early to bed and early to rise makes some of us unhealthy and sleep deprived. In Proc. of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 673--684.

Digital Library

[3]

Bruce M Altevogt, Harvey R Colten, and others. 2006. Sleep Disorders and Sleep Deprivation: An Unmet Public Health Problem. National Academies Press.

[4]

Gary Aston-Jones. 2005. Brain structures and receptors involved in alertness. Sleep medicine 6 (2005), S3--S7.

[5]

Joseph V Baranski, Ross A Pigeau, and Robert G Angus. 1994. On the ability to self-monitor cognitive performance during sleep deprivation: a calibration study. Journal of Sleep Research 3, 1 (1994), 36--44.

[6]

Christopher M Barnes and David T Wagner. 2009. Changing to daylight saving time cuts into sleep and increases workplace injuries. Journal of applied psychology 94, 5 (2009), 1305.

[7]

Mathias Basner, Daniel Mollicone, and David F Dinges. 2011. Validity and sensitivity of a brief psychomotor vigilance test (PVT-B) to total and partial sleep deprivation. Acta astronautica 69, 11 (2011), 949--959.

[8]

Mathias Basner and Joshua Rubinstein. 2011. Fitness for duty: A 3 minute version of the Psychomotor Vigilance Test predicts fatigue related declines in luggage screening performance. Journal of Occupational and Environmental Medicine 53, 10 (2011), 1146.

[9]

Frank Bentley, Konrad Tollmar, Peter Stephenson, Laura Levy, Brian Jones, Scott Robertson, Ed Price, Richard Catrambone, and Jeff Wilson. 2013. Health Mashups: Presenting statistical patterns between wellbeing data and context in natural language to promote behavior change. ACM Transactions on Computer-Human Interaction (TOCHI) 20, 5 (2013), 30.

Digital Library

[10]

Léon Bottou. 2012. Stochastic gradient descent tricks. Neural Networks: Tricks of the Trade (2012), 421--436.

[11]

Walter C Buboltz Jr, Franklin Brown, and Barlow Soper. 2001. Sleep habits and patterns of college students: a preliminary study. Journal of American College Health 50, 3 (2001), 131--135.

[12]

John A Caldwell, Melissa M Mallis, J Lynn Caldwell, Michel A Paul, James C Miller, and David F Neri. 2009. Fatigue countermeasures in aviation. Aviation, Space, and Environmental Medicine 80, 1 (2009), 29--59.

[13]

Julie Carrier and Timothy H Monk. 2000. Circadian rhythms of performance: new trends. Chronobiology international 17, 6 (2000), 719--732.

[14]

Trudie Chalder, G Berelowitz, Teresa Pawlikowska, Louise Watts, S Wessely, D Wright, and EP Wallace. 1993. Development of a fatigue scale. Journal of psychosomatic research 37, 2 (1993), 147--153.

[15]

Nancy L Digdon and Andrew J Howell. 2008. College students who have an eveningness preference report lower self-control and greater procrastination. Chronobiology international 25, 6 (2008), 1029--1046.

[16]

Angela K Dills and Rey Hernandez-Julian. 2008. Course scheduling and academic performance. Economics of Education Review 27, 6 (2008), 646--654.

[17]

David F Dinges. 1995. An overview of sleepiness and accidents. Journal of sleep research 4, s2 (1995), 4--14.

[18]

David F Dinges and John W Powell. 1985. Microcomputer analyses of performance on a portable, simple visual RT task during sustained operations. Behavior Research Methods, Instruments, & Computers 17, 6 (1985), 652--655.

[19]

Jill Dorrian, Nicole Lamond, and Drew Dawson. 2000. The ability to self-monitor performance when fatigued. Journal of Sleep Research 9, 2 (2000), 137--144.

[20]

Denzil Ferreira, Jorge Goncalves, Vassilis Kostakos, Louise Barkhuus, and Anind K Dey. 2014. Contextual experience sampling of mobile application micro-usage. In Proc. of the 16th international conference on Human-computer interaction with mobile devices & services. ACM, 91--100.

Digital Library

[21]

John F Gaski and Jeff Sagarin. 2011. Detrimental effects of daylight-saving time on SAT scores. Journal of Neuroscience, Psychology, & Economics 4, 1 (2011), 44

[22]

Namni Goel, Mathias Basner, Hengyi Rao, and David F Dinges. 2013. Circadian rhythms, sleep deprivation, and human performance. Progress in molecular biology and translational science 119 (2013), 155.

[23]

William Griesar, Daniel Zajdel, and Barry Oken. 2002. Nicotine effects on alertness and spatial attention in non-smokers. Nicotine & tobacco research 4, 2 (2002), 185--194.

[24]

Isabelle Guyon, Jason Weston, Stephen Barnhill, and Vladimir Vapnik. 2002. Gene selection for cancer classification using support vector machines. Machine learning 46, 1-3 (2002), 389--422.

Digital Library

[25]

Lynn Hasher, David Goldstein, and Cynthia May. 2005. It's About Time: Circadian Rhythms, Memory, and Aging. (2005).

[26]

Ian Hindmarch, JS Kerr, and N Sherwood. 1991. The effects of alcohol and other drugs on psychomotor performance and cognitive function. Alcohol and Alcoholism 26, 1 (1991), 71--79.

[27]

Jim A Horne and Olov Ostberg. 1975. A self-assessment questionnaire to determine morningness-eveningness in human circadian rhythms. International journal of chronobiology 4, 2 (1975), 97--110.

[28]

Peter J Huber and others. 1964. Robust estimation of a location parameter. The Annals of Mathematical Statistics 35, 1 (1964), 73--101.

[29]

Qiang Ji and Xiaojie Yang. 2002. Real-time eye, gaze, and face pose tracking for monitoring driver vigilance. Real-Time Imaging 8, 5 (2002), 357--377.

Digital Library

[30]

Thomas Kantermann, Myriam Juda, Martha Merrow, and Till Roenneberg. 2007. The human circadian clock's seasonal adjustment is disrupted by daylight saving time. Current Biology 17, 22 (2007), 1996--2000.

[31]

Matthew Kay, Kyle Rector, Sunny Consolvo, Ben Greenstein, Jacob Wobbrock, Nathaniel Watson, and Julie Kientz. 2013. PVT-touch: Adapting a reaction time test for touchscreen devices. In Pervasive Computing Technologies for Healthcare. IEEE, 248--251.

Digital Library

[32]

Paul Kelley, Steven W Lockley, Russell G Foster, and Jonathan Kelley. 2014. Synchronizing education to adolescent biology: 'let teens sleep, start school later'. Learning, Media and Technology (2014), 1--17.

[33]

N Kleitman, S Titelbaum, and P Feiveson. 1938. The effect of body temperature on reaction time. American Journal of Physiology 121, 2 (1938), 495--501.

[34]

Matthew J Kotchen and Laura E Grant. 2011. Does daylight saving time save energy? Evidence from a natural experiment in Indiana. Review of Economics and Statistics 93, 4 (2011), 1172--1185.

[35]

Tuuli A Lahti, Sami Leppämäki, Jouko Lönnqvist, and Timo Partonen. 2008. Transitions into and out of daylight saving time compromise sleep and the rest-activity cycles. BMC physiology 8, 1 (2008), 3.

[36]

Nicole Lamond and Drew Dawson. 1999. Quantifying the performance impairment associated with fatigue. Journal of sleep research 8, 4 (1999), 255--262.

[37]

Nicole Lamond, Sarah M Jay, Jillian Dorrian, Sally A Ferguson, Gregory D Roach, and Drew Dawson. 2008. The sensitivity of a palm-based psychomotor vigilance task to severe sleep loss. Behavior Research Methods 40, 1 (2008), 347--352.

[38]

Julian Lim and David F Dinges. 2008. Sleep deprivation and vigilant attention. Annals of the New York Academy of Sciences 1129, 1 (2008), 305--322.

[39]

Julian Lim and David F Dinges. 2010. A meta-analysis of the impact of short-term sleep deprivation on cognitive variables. Psychological bulletin 136, 3 (2010), 375.

[40]

Sylvia Loh, Nicole Lamond, Jill Dorrian, Gregory Roach, and Drew Dawson. 2004. The validity of psychomotor vigilance tasks of less than 10-minute duration. Behavior Research Methods 36, 2 (2004), 339--346.

[41]

Gloria Mark, Shamsi T Iqbal, Mary Czerwinski, and Paul Johns. 2014. Bored mondays and focused afternoons: The rhythm of attention and online activity in the workplace. In Proc. of the SIGCHI Conference on Human Factors in Computing Systems. 3025--3034.

Digital Library

[42]

Robert L Matchock and J Toby Mordkoff. 2009. Chronotype and time-of-day influences on the alerting, orienting, and executive components of attention. Experimental brain research 192, 2 (2009), 189--198.

[43]

Aleksandar Matic, Martin Pielot, and Nuria Oliver. 2015. Boredom-computer interaction: Boredom proneness and the use of smartphone. In Proc. of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 837--841.

Digital Library

[44]

Jun-Ki Min, Afsaneh Doryab, Jason Wiese, Shahriyar Amini, John Zimmerman, and Jason I Hong. 2014. Toss'n'turn: smartphone as sleep and sleep quality detector. In Proc. of the ACM conference on Human factors in computing systems. ACM, 477--486.

Digital Library

[45]

Timothy H Monk. 1989. A visual analogue scale technique to measure global vigor and affect. Psychiatry research 27, 1 (1989), 89--99.

[46]

Timothy H Monk, Daniel J Buysse, Charles F Reynolds, and David J Kupfer. 1996. Circadian determinants of the postlunch dip in performance. Chronobiology international 13, 2 (1996), 123--133.

[47]

Elizabeth L Murnane, Saeed Abdullah, Mark Matthews, Tanzeem Choudhury, and Geri Gay. 2015. Social (media) jet lag: How usage of social technology can modulate and reflect circadian rhythms. In Proc. of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 843--854.

Digital Library

[48]

Elizabeth L Murnane, Saeed Abdullah, Mark Matthews, Matthew Kay, Julie A Kientz, Tanzeem Choudhury, Geri Gay, and Dan Cosley. 2016. Mobile Manifestations of Alertness: Connecting Biological Rhythms with Patterns of Smartphone App Use. In Proc. of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM.

Digital Library

[49]

National Transportation Safety Board. 1999. Evaluation of U.S. Department of Transportation Efforts in the 1990s to address operator fatigue. Safety report NTSB/SR-99/01.

[50]

Sanjay R Patel, Najib T Ayas, Mark R Malhotra, David P White, Eva S Schernhammer, Frank E Speizer, Meir J Stampfer, and Frank B Hu. 2004. A prospective study of sleep duration and mortality risk in women. SLEEP 27, 3 (2004), 440--444.

[51]

Pierre Philip, Patricia Sagaspe, Jacques Taillard, Nicholas Moore, Christian Guilleminault, Montserrat Sanchez-Ortuno, Torbjörn Akerstedt, and Bernard Bioulac. 2003. Fatigue, sleep restriction, and performance in automobile drivers: a controlled study in a natural environment. SLEEP 26, 3 (2003), 277--284.

[52]

Martin Pielot, Tilman Dingler, Jose San Pedro, and Nuria Oliver. 2015. When attention is not scarce - detecting boredom from mobile phone usage. In Proc. of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 825--836.

Digital Library

[53]

Jacob Poushter. 2016. Smartphone Ownership and Internet Usage Continues to Climb in Emerging Economies. PEW.

[54]

LA Reyner, SJ Wells, V Mortlock, and JA Horne. 2012. 'Post-lunch' sleepiness during prolonged, monotonous driving --- Effects of meal size. Physiology & behavior 105, 4 (2012), 1088--1091.

[55]

Till Roenneberg, Tim Kuehnle, Peter P Pramstaller, Jan Ricken, Miriam Havel, Angelika Guth, and Martha Merrow. 2004. A marker for the end of adolescence. Current Biology 14, 24 (2004), R1038--R1039.

[56]

Till Roenneberg, Anna Wirz-Justice, and Martha Merrow. 2003. Life between clocks: daily temporal patterns of human chronotypes. Journal of biological rhythms 18, 1 (2003), 80--90.

[57]

Christina Schmidt, Fabienne Collette, Christian Cajochen, and Philippe Peigneux. 2007. A time to think: circadian rhythms in human cognition. Cognitive Neuropsychology 24, 7 (2007), 755--789.

[58]

Aaron Smith. 2015. The smartphone difference. PEW.

[59]

Neeraj Sood and Arkadipta Ghosh. 2007. The short and long run effects of daylight saving time on fatal automobile crashes. The BE Journal of Economic Analysis & Policy 7, 1 (2007).

[60]

Daniel J Taylor, Kendra C Clay, Adam D Bramoweth, Kevin Sethi, and Brandy M Roane. 2011. Circadian phase preference in college students: relationships with psychological functioning and academics. Chronobiology international 28, 6 (2011), 541--547.

[61]

HPA Van Dongen and David F Dinges. 2005. Circadian rhythms in sleepiness, alertness, and performance. In Principles and practice of sleep medicine. 4, 435--443.

[62]

Céline Vetter, Myriam Juda, and Till Roenneberg. 2012. The influence of internal time, time awake, and sleep duration on cognitive performance in shiftworkers. Chronobiology International 29, 8 (2012), 1127--1138.

[63]

David T Wagner, Christopher M Barnes, Vivien KG Lim, and D Lance Ferris. 2012. Lost sleep and cyberloafing: Evidence from the laboratory and a daylight saving time quasi-experiment. Journal of Applied Psychology 97, 5 (2012), 1068.

[64]

Nancy Wesensten, William Killgore, and Thomas Balkin. 2005. Performance and alertness effects of caffeine, dextroamphetamine, and modafinil during sleep deprivation. Journal of sleep research 14, 3 (2005), 255--266.

Cited By

Lee YSong WSong JNoh Y(2024)Extending EV Battery Lifetime: Digital Phenotyping Approach for Departure Time PredictionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997258:4(1-30)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699725
Nepal SLiu WPillai AWang WVojdanovski VHuckins JRogers CMeyer MCampbell A(2024)Capturing the College ExperienceProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435018:1(1-37)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643501
Karlgren KMcmillan D(2023)Sleep Planning with Awari: Uncovering the Materiality of Body Rhythms using Research through DesignProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581502(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581502
Show More Cited By

Index Terms

Cognitive rhythms: unobtrusive and continuous sensing of alertness using a mobile phone
1. Applied computing
  1. Life and medical sciences

Recommendations

Mobile manifestations of alertness: connecting biological rhythms with patterns of smartphone app use
MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services

Our body clock causes considerable variations in our behavioral, mental, and physical processes, including alertness, throughout the day. While much research has studied technology usage patterns, the potential impact of underlying biological processes ...
A thalamo-cortical neural mass model for the simulation of brain rhythms during sleep

Cortico-thalamic interactions are known to play a pivotal role in many brain phenomena, including sleep, attention, memory consolidation and rhythm generation. Hence, simple mathematical models that can simulate the dialogue between the cortex and the ...
Which EEG Electrodes Should Be Considered for Alertness Assessment?
CHIRA 2019: Proceedings of the 3rd International Conference on Computer-Human Interaction Research and Applications

The analysis of EEG signal is one of the objective methods used in alertness assessment. Many publications confirm the correct assessment of alertness level based on the analysis of selected brain waves. EEG registration is a difficult task; one of the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 2016

1288 pages

ISBN:9781450344616

DOI:10.1145/2971648

General Chairs:
Paul Lukowicz
DFKI and University of Kaiserslautern
,
Antonio Krüger
DFKI and Saarland University
,
Program Chairs:
Andreas Bulling
Max Planck Institute for Informatics
,
Youn-Kyung Lim
KAIST
,
Shwetak N. Patel
University of Washington

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 September 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UbiComp '16

Sponsor:

UbiComp '16: The 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 12 - 16, 2016

Germany, Heidelberg

Acceptance Rates

UbiComp '16 Paper Acceptance Rate 101 of 389 submissions, 26%;

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

65
Total Citations
View Citations
1,088
Total Downloads

Downloads (Last 12 months)82
Downloads (Last 6 weeks)6

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lee YSong WSong JNoh Y(2024)Extending EV Battery Lifetime: Digital Phenotyping Approach for Departure Time PredictionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997258:4(1-30)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699725
Nepal SLiu WPillai AWang WVojdanovski VHuckins JRogers CMeyer MCampbell A(2024)Capturing the College ExperienceProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435018:1(1-37)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643501
Karlgren KMcmillan D(2023)Sleep Planning with Awari: Uncovering the Materiality of Body Rhythms using Research through DesignProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581502(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581502
Ning ECladek ARoss MKabir SBarve AKennelly EHussain FDuffecy JLangenecker SNguyen TTulabandhula TZulueta JAjilore ODemos ALeow A(2023)Smartphone-derived Virtual Keyboard Dynamics Coupled with Accelerometer Data as a Window into Understanding Brain HealthProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580906(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580906
Kwok TKiefer PRaubal M(2023)Unobtrusive interaction: a systematic literature review and expert surveyHuman–Computer Interaction10.1080/07370024.2022.216240439:5-6(380-416)Online publication date: Feb-2023
https://doi.org/10.1080/07370024.2022.2162404
Park SZhunis AConstantinides MAiello LQuercia DCha M(2023)Social dimensions impact individual sleep quantity and qualityScientific Reports10.1038/s41598-023-36762-513:1Online publication date: 15-Jun-2023
https://doi.org/10.1038/s41598-023-36762-5
Karlgren KBrown BMcMillan D(2022)From Self-Tracking to Sleep-HackingProceedings of the ACM on Human-Computer Interaction10.1145/35556306:CSCW2(1-26)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555630
Karlgren KMcMillan D(2022)Designing for Extreme Sleepers: Rethinking the Rhythms of Sleep TechnologyNordic Human-Computer Interaction Conference10.1145/3546155.3546685(1-17)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3546155.3546685
Ponnada ALi JWang SWang WDo BDunton GIntille S(2022)Contextual Biases in Microinteraction Ecological Momentary Assessment (μEMA) Non-responseProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35172596:1(1-24)Online publication date: 29-Mar-2022
https://dl.acm.org/doi/10.1145/3517259
Yan RLiu XDutcher JTumminia MVillalba DCohen SCreswell DCreswell KMankoff JDey ADoryab A(2022)A Computational Framework for Modeling Biobehavioral Rhythms from Mobile and Wearable Data StreamsACM Transactions on Intelligent Systems and Technology10.1145/351002913:3(1-27)Online publication date: 3-Mar-2022
https://dl.acm.org/doi/10.1145/3510029
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents