research-article

Open access

An Experience Sampling Study of User Reactions to Browser Warnings in the Field

Authors:

Robert W. Reeder,

Adrienne Porter Felt,

Sunny Consolvo,

Christopher Thompson,

Serge EgelmanAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 512, Pages 1 - 13

https://doi.org/10.1145/3173574.3174086

Published: 21 April 2018 Publication History

Abstract

Web browser warnings should help protect people from malware, phishing, and network attacks. Adhering to warnings keeps people safer online. Recent improvements in warning design have raised adherence rates, but they could still be higher. And prior work suggests many people still do not understand them. Thus, two challenges remain: increasing both comprehension and adherence rates. To dig deeper into user decision making and comprehension of warnings, we performed an experience sampling study of web browser security warnings, which involved surveying over 6,000 Chrome and Firefox users in situ to gather reasons for adhering or not to real warnings. We find these reasons are many and vary with context. Contrary to older prior work, we do not find a single dominant failure in modern warning design---like habituation---that prevents effective decisions. We conclude that further improvements to warnings will require solving a range of smaller contextual misunderstandings.

References

[1]

Mustafa Acer, Emily Stark, Adrienne Porter Felt, Sascha Fahl, Radhika Bhargava, Bhanu Dev, Matt Braithwaite, Ryan Sleevi, and Parisa Tabriz. 2017. Where the Wild Warnings Are: Root Causes of Chrome Certificate Errors. In Proceedings of the 24th ACM SIGSAC Conference on Computer and Communications Security. https://research.google.com/pubs/pub46359.html

Digital Library

[2]

Devdatta Akhawe and Adrienne Porter Felt. 2013. Alice in Warningland: A Large-Scale Field Study of Browser Security Warning Effectiveness. In USENIX security symposium, Vol. 13. https://www.usenix.org/system/files/conference/ usenixsecurity13/sec13-paper_akhawe.pdf

Digital Library

[3]

Hazim Almuhimedi, Adrienne Porter Felt, Robert W. Reeder, and Sunny Consolvo. 2014. Your reputation precedes you: History, reputation, and the chrome malware warning. In Symposium on Usable Privacy and Security (SOUPS), Vol. 4. 2. https://www.usenix.org/system/files/conference/ soups2014/soups14-paper-almuhimedi.pdf

Digital Library

[4]

Bonnie Anderson, Tony Vance, Brock Kirwan, David Eargle, and Seth Howard. 2014. Users aren't (necessarily) lazy: using NeuroIS to explain habituation to security warnings. Auckland, New Zealand.

[5]

Bonnie Brinton Anderson, C. Brock Kirwan, Jeffrey L. Jenkins, David Eargle, Seth Howard, and Anthony Vance. 2015. How Polymorphic Warnings Reduce Habituation in the Brain: Insights from an fMRI Study. ACM Press, 2883--2892.

Digital Library

[6]

D. Anthony, T. Henderson, and D. Kotz. 2007. Privacy in Location-Aware Computing Environments. IEEE Pervasive Computing 6, 4 (Oct 2007), 64--72.

Digital Library

[7]

Robert Biddle, Paul C. Van Oorschot, Andrew S. Patrick, Jennifer Sobey, and Tara Whalen. 2009. Browser interfaces and extended validation SSL certificates: an empirical study. In Proceedings of the 2009 ACM workshop on Cloud computing security. ACM, 19--30. http://dl.acm.org/citation.cfm?id=1655012

Digital Library

[8]

Rainer Böhme and Stefan Köpsell. 2010. Trained to accept?: a field experiment on consent dialogs. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 2403--2406. http://dl.acm.org/citation.cfm?id=1753689

Digital Library

[9]

Cristian Bravo-Lillo, Lorrie Faith Cranor, Julie Downs, Saranga Komanduri, and Manya Sleeper. 2011. Improving Computer Security Dialogs. In Proceedings of the 13th IFIP TC 13 International Conference on Human-computer Interaction - Volume Part IV (INTERACT'11). Springer-Verlag, Berlin, Heidelberg, 18--35. http://dl.acm.org/citation.cfm?id=2042283.2042286

Digital Library

[10]

Jose Carlos Brustoloni and Ricardo Villamarin-Salomon. 2007. Improving security decisions with polymorphic and audited dialogs. In Proceedings of the 3rd symposium on Usable privacy and security. ACM, 76--85. http://dl.acm.org/citation.cfm?id=1280691

Digital Library

[11]

Juan Pablo Carrascal, Christopher Riederer, Vijay Erramilli, Mauro Cherubini, and Rodrigo de Oliveira. 2013. Your Browsing Behavior for a Big Mac: Economics of Personal Information Online. In Proceedings of the 22Nd International Conference on World Wide Web (WWW '13). ACM, New York, NY, USA, 189--200.

Digital Library

[12]

Mauro Cherubini and Nuria Oliver. 2009. A refined experience sampling method to capture mobile user experience. arXiv preprint arXiv:0906.4125 (2009).

[13]

Neil Chou, Robert Ledesma, Yuka Teraguchi, and John C. Mitchell. 2004. Client-Side Defense Against Web-Based Identity Theft. In Proceedings of the 11th Annual Network and Distributed Systems Security Symposium (NDSS '04).

[14]

Karen Church and Barry Smyth. 2009. Understanding the Intent Behind Mobile Information Needs. In Proceedings of the 14th International Conference on Intelligent User Interfaces (IUI '09). ACM, New York, NY, USA, 247--256.

Digital Library

[15]

S. Consolvo, F.R. Bentley, E.B. Hekler, and S.S. Phatak. 2017. Mobile User Research: A Practical Guide. In Synthesis Lectures on Mobile and Pervasive Computing. Morgan and Claypool Publishers, Chapter 4.

Digital Library

[16]

Sunny Consolvo, Ian E. Smith, Tara Matthews, Anthony LaMarca, Jason Tabert, and Pauline Powledge. 2005. Location Disclosure to Social Relations: Why, when,&What People Want to Share. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '05). ACM, New York, NY, USA, 81--90.

Digital Library

[17]

Rachna Dhamija, J. Doug Tygar, and Marti Hearst. 2006. Why phishing works. In Proceedings of the SIGCHI conference on Human Factors in computing systems. ACM, 581--590. http://dl.acm.org/citation.cfm?id=1124861

Digital Library

[18]

Julie S. Downs, Mandy B. Holbrook, and Lorrie Faith Cranor. 2006. Decision Strategies and Susceptibility to Phishing. In Proceedings of the Second Symposium on Usable Privacy and Security (SOUPS '06). ACM, New York, NY, USA, 79--90.

Digital Library

[19]

Serge Egelman, Lorrie Faith Cranor, and Jason Hong. 2008. You've been warned: an empirical study of the effectiveness of web browser phishing warnings. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1065--1074. http://dl.acm.org/citation.cfm?id=1357219

Digital Library

[20]

Serge Egelman and Stuart Schechter. 2013. The importance of being earnest {in security warnings}. In International Conference on Financial Cryptography and Data Security. Springer, 52--59.

[21]

Adrienne Porter Felt, Alex Ainslie, Robert W. Reeder, Sunny Consolvo, Somas Thyagaraja, Alan Bettes, Helen Harris, and Jeff Grimes. 2015. Improving SSL Warnings: Comprehension and Adherence. ACM Press, 2893--2902.

Digital Library

[22]

Adrienne Porter Felt, Robert W. Reeder, Hazim Almuhimedi, and Sunny Consolvo. 2014. Experimenting at scale with Google Chrome's SSL warning. ACM Press, 2667--2670.

Digital Library

[23]

Nathaniel Good, Rachna Dhamija, Jens Grossklags, David Thaw, Steven Aronowitz, Deirdre Mulligan, and Joseph Konstan. 2005. Stopping spyware at the gate: a user study of privacy, notice and spyware. In Proceedings of the 2005 symposium on Usable privacy and security. ACM, 43--52. http://dl.acm.org/citation.cfm?id=1073006

Digital Library

[24]

Nathaniel S. Good, Jens Grossklags, Deirdre K. Mulligan, and Joseph A. Konstan. 2007. Noticing notice: a large-scale experiment on the timing of software license agreements. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 607--616. http://dl.acm.org/citation.cfm?id=1240720

Digital Library

[25]

Jonna Häkkilä, Farnaz Vahabpour, Ashley Colley, Jani Väyrynen, and Timo Koskela. 2015. Design Probes Study on User Perceptions of a Smart Glasses Concept. In Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia (MUM '15). ACM, New York, NY, USA, 223--233.

Digital Library

[26]

Marian Harbach, Emanuel von Zezschwitz, Andreas Fichtner, Alexander De Luca, and Matthew Smith. 2014. It's a hard lock life: A field study of smartphone (un)locking behavior and risk perception. In Symposium on Usable Privacy and Security (SOUPS).

[27]

Eiji Hayashi and Jason Hong. 2011. A diary study of password usage in daily life. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2627--2630.

Digital Library

[28]

Cormac Herley. 2009. So long, and no thanks for the externalities: the rational rejection of security advice by users. In Proceedings of the 2009 workshop on New security paradigms workshop. ACM, 133--144. http://dl.acm.org/citation.cfm?id=1719050

Digital Library

[29]

Giovanni Iachello, Khai N. Truong, Gregory D. Abowd, Gillian R. Hayes, and Molly Stevens. 2006. Prototyping and Sampling Experience to Evaluate Ubiquitous Computing Privacy in the Real World. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06). ACM, New York, NY, USA, 1009--1018.

Digital Library

[30]

Philip G Inglesant and M Angela Sasse. 2010. The true cost of unusable password policies: password use in the wild. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 383--392.

Digital Library

[31]

Kat Krol, Matthew Moroz, and M. Angela Sasse. 2012. Don't work. Can't work? Why it's time to rethink security warnings. In risk and security of internet and systems (CRiSIS), 2012 7th International conference on. IEEE, 1--8. http://ieeexplore.ieee.org/abstract/document/6378951/

Digital Library

[32]

Lawrence L. Kupper and Kerry B. Hafner. 1989. On Assessing Interrater Agreement for Multiple Attribute Responses. Biometrics 45, 3 (Sept. 1989), 957.

[33]

J. Richard Landis and Gary G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics 33, 1 (March 1977), 159.

[34]

Reed Larson and Mihaly Csikszentmihalyi. 1983. The Experience Sampling Method. New Directions for Methodology of Social&Behavioral Science 15 (1983), 41--56.

[35]

Sameer Patil, Roberto Hoyle, Roman Schlegel, Apu Kapadia, and Adam J. Lee. 2015. Interrupt Now or Inform Later?: Comparing Immediate and Delayed Privacy Feedback. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1415--1418.

Digital Library

[36]

Niels Provos. 2012. Safe Browsing-Protecting Web Users for Five Years and Counting. https://www.blog.google/topics/safety-security/ safe-browsingprotecting-web-users-for/. (June 19 2012). Accessed: September 18, 2017.

[37]

Stuart E. Schechter, Rachna Dhamija, Andy Ozment, and Ian Fischer. 2007. The emperor's new security indicators. In Security and Privacy, 2007. SP'07. IEEE Symposium on. IEEE, 51--65. http://ieeexplore.ieee.org/abstract/document/4223213/

Digital Library

[38]

Fuming Shih, Ilaria Liccardi, and Daniel Weitzner. 2015. Privacy Tipping Points in Smartphones Privacy Preferences. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 807--816.

Digital Library

[39]

Manya Sleeper, Rebecca Balebako, Sauvik Das, Amber Lynn McConahy, Jason Wiese, and Lorrie Faith Cranor. 2013. The post that wasn't: exploring self-censorship on facebook. In Proceedings of the 2013 conference on Computer supported cooperative work. ACM, 793--802.

Digital Library

[40]

Andreas Sotirakopoulos, Kirstie Hawkey, and Konstantin Beznosov. 2011. On the challenges in usable security lab studies: lessons learned from replicating a study on SSL warnings. In Proceedings of the Seventh Symposium on Usable Privacy and Security. ACM, 3. http://dl.acm.org/citation.cfm?id=2078831

Digital Library

[41]

Joshua Sunshine, Serge Egelman, Hazim Almuhimedi, Neha Atri, and Lorrie Faith Cranor. 2009. Crying Wolf: An Empirical Study of SSL Warning Effectiveness. In USENIX security symposium. 399--416. http://static.usenix.org/legacy/events/sec09/tech/ full_papers/sec09_browser.pdf

Digital Library

[42]

David R. Thomas. 2006. A General Inductive Approach for Analyzing Qualitative Evaluation Data. American Journal of Evaluation 27, 2 (June 2006), 237--246.

[43]

Rick Wash and Emilee J. Rader. 2015. Too Much Knowledge? Security Beliefs and Protective Behaviors Among United States Internet Users. In SOUPS. 309--325.

[44]

Joel Weinberger and Adrienne Porter Felt. 2016. A week to remember: The impact of browser warning storage policies. In Symposium on Usable Privacy and Security. https://www.usenix.org/system/files/conference/ soups2016/soups2016-paper-weinberger.pdf

[45]

Dan Wendlandt, David G. Andersen, and Adrian Perrig. 2008. Perspectives: Improving SSH-style Host Authentication with Multi-Path Probing. In USENIX '08: Proceedings of the 2008 USENIX Annual Technical Conference. USENIX Association, Berkeley, CA, USA.

Digital Library

[46]

Primal Wijesekera, Arjun Baokar, Lynn Tsai, Joel Reardon, Serge Egelman, David Wagner, and Konstantin Beznosov. 2017. The Feasability of Dynamically Granted Permissions: Aligning Mobile Privacy with User Preferences. In Proceedings of the 2017 IEEE Symposium on Security and Privacy (Oakland '17). IEEE Computer Society.

[47]

Will Wiquist. 2016. FCC settles Verizon "supercookie" probe, requires consumer opt-in for third parties. Federal Communications Commission press release. Mar 7, 2016. https://apps.fcc.gov/edocs_public/attachmatch/ DOC-338091A1.pdf. (2016).

[48]

Min Wu, Robert C. Miller, and Simson L. Garfinkel. 2006. Do security toolbars actually prevent phishing attacks?. In Proceedings of the SIGCHI conference on Human Factors in computing systems. ACM, 601--610. http://dl.acm.org/citation.cfm?id=1124863

Digital Library

Cited By

Li KRamokapane KRashid A(2024)Usability Study of Security Features in Programmable Logic ControllersProceedings of the 2024 European Symposium on Usable Security10.1145/3688459.3688471(200-219)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3688459.3688471
Alashwali EMysuru Chandrashekar RLanyon MFaith Cranor L(2024)Detection and Impact of Debit/Credit Card Fraud: Victims' ExperiencesProceedings of the 2024 European Symposium on Usable Security10.1145/3688459.3688464(235-260)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3688459.3688464
Markert PLassak LGolla MDürmuth M(2024)Understanding Users' Interaction with Login NotificationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642823(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642823
Show More Cited By

Index Terms

An Experience Sampling Study of User Reactions to Browser Warnings in the Field

Recommendations

Improving SSL Warnings: Comprehension and Adherence
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Browsers warn users when the privacy of an SSL/TLS connection might be at risk. An ideal SSL warning would empower users to make informed decisions and, failing that, guide confused users to safety. Unfortunately, users struggle to understand and often ...
Where the Wild Warnings Are: Root Causes of Chrome HTTPS Certificate Errors
CCS '17: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security

HTTPS error warnings are supposed to alert browser users to network attacks. Unfortunately, a wide range of non-attack circumstances trigger hundreds of millions of spurious browser warnings per month. Spurious warnings frustrate users, hinder the ...
Exploring User Reactions to New Browser Cues for Extended Validation Certificates
ESORICS '08: Proceedings of the 13th European Symposium on Research in Computer Security: Computer Security

With the introduction of Extended Validation SSL certificates in Internet Explorer 7.0, web browsers are introducing new indicators to convey status information about different types of certificates. We carried out a user study which compared a proposed ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

80
Total Citations
View Citations
2,785
Total Downloads

Downloads (Last 12 months)480
Downloads (Last 6 weeks)38

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li KRamokapane KRashid A(2024)Usability Study of Security Features in Programmable Logic ControllersProceedings of the 2024 European Symposium on Usable Security10.1145/3688459.3688471(200-219)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3688459.3688471
Alashwali EMysuru Chandrashekar RLanyon MFaith Cranor L(2024)Detection and Impact of Debit/Credit Card Fraud: Victims' ExperiencesProceedings of the 2024 European Symposium on Usable Security10.1145/3688459.3688464(235-260)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3688459.3688464
Markert PLassak LGolla MDürmuth M(2024)Understanding Users' Interaction with Login NotificationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642823(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642823
Wang ZKulkarni CWilcox LTerry MMadaio M(2024)Farsight: Fostering Responsible AI Awareness During AI Application PrototypingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642335(1-40)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642335
Shrestha AFlood ASohrawardi SWright MAl-Ameen M(2024)A First Look into Targeted Clickbait and its Countermeasures: The Power of StorytellingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642301(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642301
Debnath NJain A(2024)A comprehensive survey on mobile browser security issues, challenges and solutionsInformation Security Journal: A Global Perspective10.1080/19393555.2024.2347256(1-20)Online publication date: 29-Apr-2024
https://doi.org/10.1080/19393555.2024.2347256
Atashpanjeh HPaudel RAl-Ameen M(2024)Mental Model-Based Designs: The Study in Privacy Policy LandscapeInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2392064(1-20)Online publication date: 2-Oct-2024
https://doi.org/10.1080/10447318.2024.2392064
Baltuttis DTeubner T(2024)Effects of visual risk indicators on phishing detection behaviorComputers and Security10.1016/j.cose.2024.103940144:COnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.cose.2024.103940
Zheng SBecker IKelley PKapadia A(2023)Checking, nudging or scoring? evaluating e-mail user security tools76Proceedings of the Nineteenth USENIX Conference on Usable Privacy and Security10.5555/3632186.3632190(57-76)Online publication date: 7-Aug-2023
https://dl.acm.org/doi/10.5555/3632186.3632190
Stegman JTrottier PHillier CKhan HMannan MCalandrino JTroncoso C(2023)"My privacy for their security"Proceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620438(3583-3600)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620438
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents