research-article

Implications of XR on Privacy, Security and Behaviour: Insights from Experts

Authors:

Melvin Abraham,

Mohamed KhamisAuthors Info & Claims

NordiCHI '22: Nordic Human-Computer Interaction Conference

Article No.: 30, Pages 1 - 12

https://doi.org/10.1145/3546155.3546691

Published: 08 October 2022 Publication History

Abstract

Extended-Reality (XR) devices are packed with sensors that allow tracking of users (e.g., behaviour, actions, eye-gaze) and their surroundings (e.g., people, places, objects). As a consequence, XR devices pose significant risks to privacy, security, and our ability to understand and influence the behaviour of users - risks that will be amplified by ever-increasing adoption. This necessitates addressing these concerns before XR becomes ubiquitous. We conducted three focus groups with thirteen XR experts from industry and academia interested in XR, security, and privacy, to investigate current and emerging issues relating to security, privacy, and influencing behaviour. We identified issues such as virtual threats leading to physical harm, missing opting-out methods, and amplifying bias through perceptual filters. From the results we establish a collection of prescient challenges relating to security, privacy and behavioural manipulation within XR and present recommendations working towards developing future XR devices that better support security and privacy by default.

References

[1]

Article 19. 2021. Emotion Recognition Technology Report. https://www.article19.org/emotion-recognition-technology-report/

[2]

Fehmi Ben Abdesslem, Tristan Henderson, Sacha Brostoff, and M Angela Sasse. 2011. Context-based personalised settings for mobile location sharing. (2011).

[3]

Devon Adams, Alseny Bah, Catherine Barwulor, Nureli Musaby, Kadeem Pitkin, and Elissa M Redmiles. 2018. Ethics emerging: the story of privacy and security perceptions in virtual reality. In Fourteenth Symposium on Usable Privacy and Security ({SOUPS} 2018). 427–442.

Digital Library

[4]

Annie I Antón, Julia Brande Earp, Qingfeng He, William Stufflebeam, Davide Bolchini, and Carlos Jensen. 2004. Financial privacy policies and the need for standardization. IEEE Security & privacy 2, 2 (2004), 36–45.

Digital Library

[5]

Apple. 2022. About the orange and green indicators in your iPhone status bar. https://support.apple.com/en-gb/HT211876

[6]

Susanne Barth, Dan Ionita, and Pieter Hartel. 2022. Understanding Online Privacy—A Systematic Review of Privacy Visualizations and Privacy by Design Guidelines. ACM Computing Surveys (CSUR) 55, 3 (2022), 1–37.

Digital Library

[7]

Mitchell Baxter, Anna Bleakley, Justin Edwards, Leigh Clark, Benjamin R Cowan, and Julie R Williamson. 2021. “You, Move There!”: Investigating the Impact of Feedback on Voice Control in Virtual Environments. In CUI 2021-3rd Conference on Conversational User Interfaces. 1–9.

[8]

Shlomo Berkovsky, Ronnie Taib, Irena Koprinska, Eileen Wang, Yucheng Zeng, Jingjie Li, and Sabina Kleitman. 2019. Detecting personality traits using eye-tracking data. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[9]

Ann Blandford, Dominic Furniss, and Stephann Makri. 2016. Qualitative HCI research: Going behind the scenes. Synthesis lectures on human-centered informatics 9, 1(2016), 1–115.

[10]

Cristina Botella, Javier Fernández-Álvarez, Verónica Guillén, Azucena García-Palacios, and Rosa Baños. 2017. Recent progress in virtual reality exposure therapy for phobias: a systematic review. Current psychiatry reports 19, 7 (2017), 1–13.

[11]

Andreas Bulling and Thorsten O. Zander. 2014. Cognition-Aware Computing. IEEE Pervasive Computing 13, 3 (2014), 80–83. https://doi.org/10.1109/MPRV.2014.42

[12]

Shreya Chopra and Frank Maurer. 2020. Evaluating User Preferences for Augmented Reality Interactions with the Internet of Things. In Proceedings of the International Conference on Advanced Visual Interfaces. 1–9.

Digital Library

[13]

Jaybie A De Guzman, Kanchana Thilakarathna, and Aruna Seneviratne. 2019. Security and privacy approaches in mixed reality: A literature survey. ACM Computing Surveys (CSUR) 52, 6 (2019), 1–37.

Digital Library

[14]

Tamara Denning, Zakariya Dehlawi, and Tadayoshi Kohno. 2014. In situ with bystanders of augmented reality glasses: Perspectives on recording and privacy-mediating technologies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2377–2386.

Digital Library

[15]

Ellysse Dick. 2020. How to Address Privacy Questions Raised by the Expansion of Augmented Reality in Public Spaces. Technical Report. Information Technology and Innovation Foundation. https://itif.org/publications/2020/12/14/how-address-privacy-questions-raised-expansion-augmented-reality-public

[16]

Myrthe Faber, Robert Bixler, and Sidney K D’Mello. 2018. An automated behavioral measure of mind wandering during computerized reading. Behavior Research Methods 50, 1 (2018), 134–150.

[17]

Jan Fernback and Zizi Papacharissi. 2007. Online privacy as legal safeguard: the relationship among consumer, online portal, and privacy policies. New Media & Society 9, 5 (2007), 715–734.

[18]

Simone Fischer-Hübner and Harald Zwingelberg. 2010. UI Prototypes: Policy Administration and Presentation Version 2. http://primelife.ercim.eu/

[19]

Grace Fox, Colin Tonge, Theo Lynn, and John Mooney. 2018. Communicating compliance: developing a GDPR privacy label. (2018).

[20]

Jan Gugenheimer, Christian Mai, Mark McGill, Julie Williamson, Frank Steinicke, and Ken Perlin. 2019. Challenges using head-mounted displays in shared and social spaces. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–8.

Digital Library

[21]

Jassim Happa, Anthony Steed, and Mashhuda Glencross. 2021. Privacy-certification standards for extended-reality devices and services. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 397–398.

[22]

Brittan Heller. 2020. Reimagining Reality: Human Rights and Immersive Technology. Carr Center Discussion Paper Series2020-008 (2020).

[23]

Steven Hickson, Nick Dufour, Avneesh Sud, Vivek Kwatra, and Irfan Essa. 2019. Eyemotion: Classifying facial expressions in VR using eye-tracking cameras. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1626–1635.

[24]

Sabrina Hoppe, Tobias Loetscher, Stephanie A Morey, and Andreas Bulling. 2018. Eye movements during everyday behavior predict personality traits. Frontiers in human neuroscience(2018), 105.

[25]

J Thomas Hutton, JA Nagel, and Ruth B Loewenson. 1984. Eye tracking dysfunction in Alzheimer-type dementia. Neurology 34, 1 (1984), 99–99.

[26]

Marcello Ienca. 2017. Do We Have a Right to Mental Privacy and Cognitive Liberty?https://blogs.scientificamerican.com/observations/do-we-have-a-right-to-mental-privacy-and-cognitive-liberty/

[27]

XR Safety Initiative. 2020. The XRSI privacy framework. (2020).

[28]

Carlos Jensen and Colin Potts. 2004. Privacy policies as decision-making tools: an evaluation of online privacy notices. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. 471–478.

Digital Library

[29]

Christina Katsini, Yasmeen Abdrabou, George Raptis, Mohamed Khamis, and Florian Alt. 2020. The Role of Eye Gaze in Security and Privacy Applications: Survey and Future HCI Research Directions. In Proceedings of the 38th Annual ACM Conference on Human Factors in Computing Systems (Honolulu, Hawaii, USA) (CHI ’20). ACM, New York, NY, USA, 21 pages. https://doi.org/10.1145/3313831.3376840

Digital Library

[30]

Patrick Gage Kelley, Joanna Bresee, Lorrie Faith Cranor, and Robert W Reeder. 2009. A” nutrition label” for privacy. In Proceedings of the 5th Symposium on Usable Privacy and Security. 1–12.

Digital Library

[31]

Jenny Kitzinger. 1995. Qualitative research: introducing focus groups. Bmj 311, 7000 (1995), 299–302.

[32]

Craig A Kuechenmeister, Patrick H Linton, Thelma V Mueller, and Hilton B White. 1977. Eye tracking in relation to age, sex, and illness. Archives of General Psychiatry 34, 5 (1977), 578–579.

[33]

Daniel J Liebling and Sören Preibusch. 2014. Privacy considerations for a pervasive eye tracking world. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication. 1169–1177.

Digital Library

[34]

Philipp Mayring. 2014. Qualitative content analysis: theoretical foundation, basic procedures and software solution. (2014).

[35]

Mark McGill. 2021. The IEEE Global Initiative on Ethics of Extended Reality (XR) Report–Extended Reality (XR) and the Erosion of Anonymity and Privacy. (2021), 24.

[36]

Sam Meenasian. 2015. How Wearable Technology Benefits Health Insurance Companies. https://www.businessinsuranceusa.com/news/technology-related/wearable-technology-benefits-health-insurance-companies/

[37]

Matthias Mehldau. 2007. Iconset for data-privacy declarations v0.1,. https://netzpolitik.org/wp-upload/data-privacy-icons-v01.pdf

[38]

Abraham Hani Mhaidli and Florian Schaub. 2021. Identifying manipulative advertising techniques in xr through scenario construction. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–18.

Digital Library

[39]

Michael Middleton. 2022. The IEEE Global Initiative on Ethics of Extended Reality (XR) Report–Business, Finance, and Economics. The IEEE Global Initiative on Ethics of Extended Reality (XR) Report–Business, Finance, and Economics (March 2022), 1–30.

[40]

Paul Milgram, Haruo Takemura, Akira Utsumi, and Fumio Kishino. 1995. Augmented reality: A class of displays on the reality-virtuality continuum. In Telemanipulator and telepresence technologies, Vol. 2351. International Society for Optics and Photonics, 282–292.

[41]

Mark Roman Miller, Fernanda Herrera, Hanseul Jun, James A Landay, and Jeremy N Bailenson. 2020. Personal identifiability of user tracking data during observation of 360-degree VR video. Scientific Reports 10, 1 (2020), 1–10.

[42]

Alec G Moore, Ryan P McMahan, Hailiang Dong, and Nicholas Ruozzi. 2021. Personal Identifiability of User Tracking Data During VR Training. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 556–557.

[43]

Helen Nissenbaum. 1998. Protecting privacy in an information age: The problem of privacy in public. Law and philosophy (1998), 559–596.

[44]

Helen Nissenbaum. 2011. A contextual approach to privacy online. Daedalus 140, 4 (2011), 32–48.

[45]

Parmy Olson. 2014. Wearable Tech Is Plugging Into Health Insurance. https://www.forbes.com/sites/parmyolson/2014/06/19/wearable-tech-health-insurance/?sh=41a80a2d18bd

[46]

Joseph O’Hagan, Julie R Williamson, Mark McGill, and Mohamed Khamis. 2021. Safety, Power Imbalances, Ethics and Proxy Sex: Surveying In-The-Wild Interactions Between VR Users and Bystanders. In 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 211–220.

[47]

Sarah Prange, Ahmed Shams, Robin Piening, Yomna Abdelrahman, and Florian Alt. 2021. Priview–exploring visualisations to support users’ privacy awareness. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–18.

Digital Library

[48]

Franziska Roesner, Tadayoshi Kohno, and David Molnar. 2014. Security and Privacy for Augmented Reality Systems. Commun. ACM 57, 4 (apr 2014), 88–96. https://doi.org/10.1145/2580723.2580730

Digital Library

[49]

Arianna Rossi and Monica Palmirani. 2017. A Visualization Approach for Adaptive Consent in the European Data Protection Framework. In 2017 Conference for E-Democracy and Open Government (CeDEM). 159–170. https://doi.org/10.1109/CeDEM.2017.23

[50]

Takahito Sakamoto and Masahiro Matsunaga. 2019. After GDPR, still tracking or not? Understanding opt-out states for online behavioral advertising. In 2019 IEEE Security and Privacy Workshops (SPW). IEEE, 92–99.

[51]

Florian Schaub, Rebecca Balebako, Adam L Durity, and Lorrie Faith Cranor. 2015. A design space for effective privacy notices. In Eleventh symposium on usable privacy and security (SOUPS 2015). 1–17.

Digital Library

[52]

Terence Sim, Sheng Zhang, Rajkumar Janakiraman, and Sandeep Kumar. 2007. Continuous verification using multimodal biometrics. IEEE transactions on pattern analysis and machine intelligence 29, 4(2007), 687–700.

Digital Library

[53]

Maximilian Speicher, Brian D Hall, and Michael Nebeling. 2019. What is mixed reality?. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–15.

Digital Library

[54]

Julian Steil, Inken Hagestedt, Michael Xuelin Huang, and Andreas Bulling. 2019. Privacy-aware eye tracking using differential privacy. In Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications. 1–9.

Digital Library

[55]

Wen-Jie Tseng, Elise Bonnail, Mark Mcgill, Mohamed Khamis, Eric Lecolinet, Samuel Huron, and Jan Gugenheimer. 2022. The Dark Side of Perceptual Manipulations in Virtual Reality. In CHI Conference on Human Factors in Computing Systems (CHI’22).

[56]

Bibi Van den Berg and Simone Van der Hof. 2012. What happens to my data? A novel approach to informing users of data processing practices. First Monday 17, 7 (2012).

[57]

Max Van Kleek, Ilaria Liccardi, Reuben Binns, Jun Zhao, Daniel J Weitzner, and Nigel Shadbolt. 2017. Better the devil you know: Exposing the data sharing practices of smartphone apps. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 5208–5220.

Digital Library

[58]

Radu-Daniel Vatavu, Pejman Saeghe, Teresa Chambel, Vinoba Vinayagamoorthy, and Marian F Ursu. 2020. Conceptualizing Augmented Reality Television for the Living Room. In ACM International Conference on Interactive Media Experiences (Cornella, Barcelona, Spain) (IMX ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3391614.3393660

Digital Library

[59]

T Franklin Waddell, Joshua R Auriemma, and S Shyam Sundar. 2016. Make it simple, or force users to read? Paraphrased design improves comprehension of end user license agreements. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 5252–5256.

Digital Library

[60]

Katrin Wolf, Karola Marky, and Markus Funk. 2018. We should start thinking about privacy implications of sonic input in everyday augmented reality!Mensch und Computer 2018-Workshopband(2018).

Cited By

Gupta BHan CChaurasia P(2024)Privacy-Preserving Techniques and User Anonymity in the MetaverseMetaverse Security Paradigms10.4018/979-8-3693-3824-7.ch012(304-317)Online publication date: 21-Aug-2024
https://doi.org/10.4018/979-8-3693-3824-7.ch012
Oh SShon T(2024)Digital Forensics for Analyzing Cyber Threats in the XR Technology Ecosystem within Digital TwinsElectronics10.3390/electronics1313265313:13(2653)Online publication date: 6-Jul-2024
https://doi.org/10.3390/electronics13132653
Hallal LRhinelander JVenkat R(2024)Recent Trends of Authentication Methods in Extended Reality: A SurveyApplied System Innovation10.3390/asi70300457:3(45)Online publication date: 28-May-2024
https://doi.org/10.3390/asi7030045
Show More Cited By

Index Terms

Implications of XR on Privacy, Security and Behaviour: Insights from Experts
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
2. Security and privacy
  1. Human and societal aspects of security and privacy

Recommendations

Novel Challenges of Safety, Security and Privacy in Extended Reality
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

Extended Reality (AR/VR/MR) technology is becoming increasingly affordable and capable, becoming ever more interwoven with everyday life. HCI research has focused largely on innovation around XR technology, exploring new use cases and interaction ...
What is XR? Towards a Framework for Augmented and Virtual Reality
Abstract
Augmented Reality (AR), Virtual Reality (VR), Mixed Reality, and Extended Reality (often – misleadingly – abbreviated as XR) are commonly used terms to describe how technologies generate or modify reality. However, academics and ...
Highlights
- XR should not be used as an abbreviation for “Extended Reality”; X should be the placeholder for “all” new reality formats.
Exploring User Behaviour in Asymmetric Collaborative Mixed Reality
VRST '22: Proceedings of the 28th ACM Symposium on Virtual Reality Software and Technology

A common issue for collaborative mixed reality is the asymmetry of interaction with the shared virtual environment. For example, an augmented reality (AR) user might use one type of head-mounted display (HMD) in a physical environment, while a virtual ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

NordiCHI '22: Nordic Human-Computer Interaction Conference

October 2022

1091 pages

ISBN:9781450396998

DOI:10.1145/3546155

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Engineering and Physical Sciences Research Council

Conference

NordiCHI '22

NordiCHI '22: Nordic Human-Computer Interaction Conference

October 8 - 12, 2022

Aarhus, Denmark

Acceptance Rates

Overall Acceptance Rate 379 of 1,572 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
642
Total Downloads

Downloads (Last 12 months)392
Downloads (Last 6 weeks)39

Reflects downloads up to 28 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gupta BHan CChaurasia P(2024)Privacy-Preserving Techniques and User Anonymity in the MetaverseMetaverse Security Paradigms10.4018/979-8-3693-3824-7.ch012(304-317)Online publication date: 21-Aug-2024
https://doi.org/10.4018/979-8-3693-3824-7.ch012
Oh SShon T(2024)Digital Forensics for Analyzing Cyber Threats in the XR Technology Ecosystem within Digital TwinsElectronics10.3390/electronics1313265313:13(2653)Online publication date: 6-Jul-2024
https://doi.org/10.3390/electronics13132653
Hallal LRhinelander JVenkat R(2024)Recent Trends of Authentication Methods in Extended Reality: A SurveyApplied System Innovation10.3390/asi70300457:3(45)Online publication date: 28-May-2024
https://doi.org/10.3390/asi7030045
Cools RVenema REsteves ASimeone A(2024)The Impact of Near-Future Mixed Reality Contact Lenses on Users' Lives via an Immersive Speculative Enactment and Focus GroupsProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682099(1-13)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682099
Krauß VSaeghe PBoden AKhamis MMcGill MGugenheimer JNebeling M(2024)What Makes XR Dark? Examining Emerging Dark Patterns in Augmented and Virtual Reality through Expert Co-DesignACM Transactions on Computer-Human Interaction10.1145/366034031:3(1-39)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3660340
Abraham MMcgill MKhamis M(2024)What You Experience is What We Collect: User Experience Based Fine-Grained Permissions for Everyday Augmented RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642668(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642668
Fiani CBretin RMacdonald SKhamis MMcgill M(2024)"Pikachu would electrocute people who are misbehaving": Expert, Guardian and Child Perspectives on Automated Embodied Moderators for Safeguarding Children in Social Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642144(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642144
Hadan HWang DNacke LZhang-Kennedy L(2024)Privacy in Immersive Extended Reality: Exploring User Perceptions, Concerns, and Coping StrategiesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642104(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642104
Tukur MSchneider JHouseh MDokoro AIsmail UDawaki MAgus M(2024)The Metaverse digital environmentsJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2024.10196736:2Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.jksuci.2024.101967
Hine ERezende IRoberts HWong DTaddeo MFloridi L(2024)Safety and Privacy in Immersive Extended Reality: An Analysis and Policy RecommendationsDigital Society10.1007/s44206-024-00114-13:2Online publication date: 3-Jul-2024
https://doi.org/10.1007/s44206-024-00114-1
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents