research-article

Open access

A Call for Auditable Virtual, Augmented and Mixed Reality

Authors:

Richard Cloete,

Jatinder SinghAuthors Info & Claims

VRST '20: Proceedings of the 26th ACM Symposium on Virtual Reality Software and Technology

Article No.: 16, Pages 1 - 6

https://doi.org/10.1145/3385956.3418960

Published: 01 November 2020 Publication History

All formats PDF

Abstract

XR (Virtual, Augmented and Mixed Reality) technologies are growing in prominence. However, they are increasingly being used in sectors and in situations that can result in harms. As such, this paper argues the need for auditability to become a key consideration of XR systems. Auditability entails capturing information of a system’s operation to enable oversight, inspection or investigation. Things can and will go wrong, and information that helps unpack situations of failure or harm, and that enables accountability and recourse, will be crucial to XR’s adoption and acceptance. In drawing attention to the urgent need for auditability, we illustrate some risks associated with XR technology and their audit implications, and present some initial findings from a survey with developers indicating the current ‘haphazard’ approach towards such concerns. We also highlight some challenges and considerations of XR audit in practice, as well as areas of future work for taking this important area of research forward.

References

[1]

Hasaneen Fathy Al Janabi, Abdullatif Aydin, Sharanya Palaneer, Nicola Macchione, Ahmed Al-Jabir, Muhammad Shamim Khan, Prokar Dasgupta, and Kamran Ahmed. 2019. Effectiveness of the HoloLens mixed-reality headset in minimally invasive surgery: a simulation-based feasibility study. Surgical Endoscopy 34, 3 (2019), 1143–1149.

[2]

Philip D. Austin and Philip J. Siddall. 2019. Virtual reality for the Treatment of Neuropathic Pain in People with Spinal Cord Injuries: A scoping review. The Journal of Spinal Cord Medicine 0, 0 (2019), 1–11. https://doi.org/10.1080/10790268.2019.1575554 30707649.

[3]

Ronald Azuma and Chris Furmanski. 2003. Evaluating Label Placement for Augmented Reality View Management. In Proceedings of the 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality(ISMAR ’03). IEEE Computer Society, USA, 66.

Digital Library

[4]

Stefania Barbieri, Gianna Vettore, Vincenzo Pietrantonio, Rossella Snenghi, Alberto Tredese, Mauro Bergamini, Sara Previato, Armando Stefanati, Rosa Maria Gaudio, and Paolo Feltracco. 2017. Pedestrian Inattention Blindness While Playing Pokémon Go as an Emerging Health-Risk Behavior: A Case Report. Journal of medical Internet research 19, 4 (April 2017), e86. https://doi.org/10.2196/jmir.6596

[5]

Lindsay Blackwell, Nicole Ellison, Natasha Elliott-Deflo, and Raz Schwartz. 2019. Harassment in Social Virtual Reality. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–25. https://doi.org/10.1145/3359202

Digital Library

[6]

ClassVR. 2020. ClassVR - Virtual Reality for the classroom. https://www.classvr.com/virtual-reality-in-education/. Accessed: 2020-06-02.

[7]

Richard Cloete, Chris Norval, and Jatinder Singh. 2020. A Call for XR Auditability – Survey Specifics. https://github.com/compacctsys/A-call-for-XR-auditability.

[8]

Edmanuel Cruz, Sergio Orts-Escolano, Francisco Gomez-Donoso, Carlos Rizo, Jose Carlos Rangel, Higinio Mora, and Miguel Cazorla. 2018. An augmented reality application for improving shopping experience in large retail stores. Virtual Reality 23, 3 (2018), 281–291. https://doi.org/10.1007/s10055-018-0338-3

Digital Library

[9]

Jaybie A. De Guzman, Kanchana Thilakarathna, and Aruna Seneviratne. 2019. Security and Privacy Approaches in Mixed Reality: A Literature Survey. ACM Comput. Surv. 52, 6 (2019). https://doi.org/10.1145/3359626

Digital Library

[10]

Mohammed S. Elbamby, Cristina Perfecto, Mehdi Bennis, and Klaus Doppler. 2018. Toward Low-Latency and Ultra-Reliable Virtual Reality. IEEE Network 32, 2 (2018), 78–84. https://doi.org/10.1109/MNET.2018.1700268

[11]

Mara Faccio and John J. McConnell. 2020. Death by Pokémon GO: The Economic and Human Cost of Using Apps While Driving. Journal of Risk and Insurance 87, 3 (2020), 815–849. https://doi.org/10.1111/jori.12301 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/jori.12301

[12]

Ben Gilbert. 2016. This is the first major car crash caused by a person playing ‘Pokémon Go’. https://www.businessinsider.com/first-car-crash-caused-by-pokemon-go-2016-7?r=US&IR=T.

[13]

Jennifer Herron. 2016. Augmented Reality in Medical Education and Training. Journal of Electronic Resources in Medical Libraries 13, 2 (2016), 51–55. https://doi.org/10.1080/15424065.2016.1175987

[14]

Sean Hollister. 2019. Here’s the US Army version of HoloLens that Microsoft employees were protesting. https://www.theverge.com/2019/4/6/18298335/microsoft-hololens-us-military-version. Accessed: 2020-01-31.

[15]

L. Hou, Y. Wang, X. Wang, N. Maynard, I. T. Cameron, S. Zhang, and Y. Jiao. 2014. Combining Photogrammetry and Augmented Reality Towards an Integrated Facility Management System for the Oil Industry. Proc. IEEE 102, 2 (2014), 204–220.

[16]

HTC. 2020. HTC Vive Health and Safety. https://dl4.htc.com/vive/safty_guide/91H02887-08M Rev.A.PDF. Accessed: 2020-03-07.

[17]

C. E. Hughes, C. B. Stapleton, D. E. Hughes, and E. M. Smith. 2005. Mixed reality in education, entertainment, and training. IEEE Computer Graphics and Applications 25, 6 (2005), 24–30.

Digital Library

[18]

Wee Khor, B. Baker, Kavit Amin, Adrian Chan, Ketan Patel, and Jason Wong. 2016. Augmented and Virtual Reality in Surgery - The Digital Surgical Environment: Applications, Limitations and Legal Pitfalls. Annals of Translational Medicine 4 (12 2016), 454–454. https://doi.org/10.21037/atm.2016.12.23

[19]

Kiron Lebeck, Kimberly Ruth, Tadayoshi Kohno, and Franziska Roesner. 2017. Securing Augmented Reality Output. In 2017 IEEE Symposium on Security and Privacy (SP). IEEE, San Jose, CA, USA, 320–337. https://doi.org/10.1109/SP.2017.13

[20]

X. Li and Y. Chen. 2020. Auto-Hierarchical Data Algorithm: Focus on Increasing Users’ Motivation and Duration In Virtual Reality. In 2020 5th IEEE International Conference on Big Data Analytics (ICBDA). 150–153.

[21]

Jessica Martin. 2019. Oil and Gas Augmented Reality Solutions: AR Oil and Gas. https://upskill.io/skylight/industries/oil-and-gas/.

[22]

Microsoft. 2020. Safety Information. https://support.microsoft.com/en-gb/help/4023454/safety-information. Accessed: 2020-03-07.

[23]

A.Y.C. Nee and S.K. Ong. 2013. Virtual and Augmented Reality Applications in Manufacturing. IFAC Proceedings Volumes 46, 9 (2013), 15–26. https://doi.org/10.3182/20130619-3-RU-3018.00637 7th IFAC Conference on Manufacturing Modelling, Management, and Control.

[24]

Chris Norval, Jennifer Cobbe, and Jatinder Singh. 2020. Towards an accountable Internet of Things: A call for ‘reviewability’. In Privacy by Design for the Internet of Things: Building Accountability and Security (to appear). The Institution of Engineering and Technology.

[25]

Oculus. 2020. Oculus Health and Safety Warnings. https://securecdn.oculus.com/sr/oculusquest-warning-english. Accessed: 2020-03-07.

[26]

S. K. Ong, M. L. Yuan, and A. Y. C. Nee. 2008. Augmented Reality Applications in Manufacturing: A Survey. International Journal of Production Research 46, 10 (2008), 2707–2742. https://doi.org/10.1080/00207540601064773

[27]

Pixabay. 2020. Buildings City Japan - Free photo on Pixabay. https://pixabay.com/photos/buildings-city-japan-street-tokyo-4060467/. Accessed: 2020-06-14.

[28]

Gabriele Pizzi, Daniele Scarpi, Marco Pichierri, and Virginia Vannucci. 2019. Virtual Reality, Real Reactions?: Comparing Consumers’ Perceptions and Shopping Orientation Across Physical and Virtual-Reality Retail Stores. Computers in Human Behavior 96 (2019), 1–12. https://doi.org/10.1016/j.chb.2019.02.008

Digital Library

[29]

Pokémon GO Death Tracker. 2020. Pokémon GO Death Tracker. http://pokemongodeathtracker.com/. Accessed: 2020-06-02.

[30]

Wendy Powell, Tom Alexander Garner, Seth Shapiro, and Bryce Paul. 2017. Virtual Reality In Entertainment: The State Of The Industry.

[31]

Kjetil Raaen and Ivar Kjellmo. 2015. Measuring Latency in Virtual Reality Systems. In Entertainment Computing - ICEC 2015, Konstantinos Chorianopoulos, Monica Divitini, Jannicke Baalsrud Hauge, Letizia Jaccheri, and Rainer Malaka(Eds.). Cham, 457–462.

Digital Library

[32]

Kate Richards, Kai Wong, and Mansoor Khan. 2017. Augmented reality game related injuries. New Horizons in Clinical Case Reports 2 (2017), 27.

[33]

RT. 2016. Russian Military Testing 1st-Ever VR Helmet for Drone Pilots. https://www.rt.com/news/365466-russia-vr-helmet-drone/. Accessed: 2020-01-31.

[34]

E. E. Sabelman and R. Lam. 2015. The real-life dangers of augmented reality. IEEE Spectrum 52, 7 (2015), 48–53.

Digital Library

[35]

Christian Sandor, Martin Fuchs, Alvaro Cassinelli, Hao Li, Richard Newcombe, Goshiro Yamamoto, and Steven Feiner. 2015. Breaking the Barriers to True Augmented Reality. arXiv (2015).

[36]

[36] OpenGL SC.2011. https://www.khronos.org/openglsc/. Accessed: 2020-08-24.

[37]

J. Singh, J. Cobbe, and C. Norval. 2019. Decision Provenance: Harnessing Data Flow for Accountable Systems. IEEE Access 7(2019), 6562–6574.

[38]

J. Singh, T. Pasquier, J. Bacon, H. Ko, and D. Eyers. 2016. Twenty Security Considerations for Cloud-Supported Internet of Things. IEEE Internet of Things Journal 3, 3 (2016), 269–284.

[39]

UK Government. 2020. Online Harms White Paper. https://www.gov.uk/government/consultations/online-harms-white-paper. Accessed: 2020-03-23.

[40]

G. Stewart Von Itzstein, Mark Billinghurst, Ross T. Smith, and Bruce H. Thomas. 2017. Augmented Reality Entertainment: Taking Gaming Out of the Box. In Encyclopedia of Computer Graphics and Games. Springer International Publishing, Cham, 1–9.

[41]

Xiangyu Wang, Martijn Truijens, Lei Hou, Ying Wang, and Ying Zhou. 2014. Integrating Augmented Reality with Building Information Modeling: Onsite Construction Process Controlling for Liquefied Natural Gas Industry. Automation in Construction 40 (2014), 96–105.

[42]

Colin Ware and Ravin Balakrishnan. 1994. Reaching for Objects in VR Displays: Lag and Frame Rate. ACM Trans. Comput.-Hum. Interact. 1, 4 (1994), 331–356.

Digital Library

[43]

W. Xu, H. Liang, Y. Chen, X. Li, and K. Yu. 2020. Exploring Visual Techniques for Boundary Awareness During Interaction in Augmented Reality Head-Mounted Displays. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 204–211.

[44]

Shu Ching Yang. 2012. Paths to Bullying in Online Gaming: The Effects of Gender, Preference for Playing Violent Games, Hostility, and Aggressive Behavior on Bullying. Journal of Educational Computing Research 47, 3 (2012), 235–249.

[45]

Z. Zhu, S. German, and I. Brilakis. 2010. Detection of Large-scale Concrete Columns for Automated Bridge Inspection. Automation in Construction 19, 8 (2010), 1047–1055. The role of VR and BIM to manage the construction and design processes.

Cited By

Effing RHinz M(2024)Are Children Ready for the Metaverse? The Minefield of Virtual Participation in Digital Social Spaces with Harmful Content and BehaviorElectronic Participation10.1007/978-3-031-70804-6_1(1-15)Online publication date: 25-Aug-2024
https://doi.org/10.1007/978-3-031-70804-6_1
Norval CCloete RSingh J(2023)Navigating the Audit Landscape: A Framework for Developing Transparent and Auditable XRProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3594090(1418-1431)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3594090
Venugopal JSubramanian APeatchimuthu J(2023)The realm of metaverse: A surveyComputer Animation and Virtual Worlds10.1002/cav.215034:5Online publication date: 2-Mar-2023
https://doi.org/10.1002/cav.2150
Show More Cited By

Recommendations

Navigating the Audit Landscape: A Framework for Developing Transparent and Auditable XR
FAccT '23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency

“Extended reality” (XR) systems work to blend the physical and digital worlds. This means that XR is highly contextual: its functionality, operation and therefore consequences are driven by a tight, run-time coupling of the technology, the user, and ...
Auditable Augmented/Mixed/Virtual Reality: The Practicalities of Mobile System Transparency

Virtual, Augmented and Mixed Reality (XR) technologies are becoming increasingly pervasive. However, the contextual nature of XR, and its tight coupling of the digital and physical environments, brings real propensity for loss and harm.

This means that ...
Auditable zerocoin scheme with user awareness
ICCSP '19: Proceedings of the 3rd International Conference on Cryptography, Security and Privacy

With the development of Blockchain, more and more distributed cryptocurrencies continue to emerge. Most of these currencies use cryptographic techniques (such as zero-knowledge proofs, ring signatures, etc.) to enhance privacy protection, but at the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

VRST '20: Proceedings of the 26th ACM Symposium on Virtual Reality Software and Technology

November 2020

429 pages

ISBN:9781450376198

DOI:10.1145/3385956

Editors:
Robert J. Teather
Carleton University, Canada
,
Chris Joslin
Carleton University, Canada
,
Wolfgang Stuerzlinger
Simon Fraser University, Canada
,
Pablo Figueroa
Universidad de los Andes, Colombia
,
Yaoping Hu
University of Calgary, Canada
,
Anil Ufuk Batmaz
Simon Fraser University, Canada
,
Wonsook Lee
University of Ottawa, Canada
,
Francisco Ortega
Colorado State University, USA

Copyright © 2020 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2020

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

EPSRC

Conference

VRST '20

Sponsor:

VRST '20: 26th ACM Symposium on Virtual Reality Software and Technology

November 1 - 4, 2020

Virtual Event, Canada

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Upcoming Conference

VRST '24

Sponsor:
sigchi
sigchi

30th ACM Symposium on Virtual Reality Software and Technology

October 9 - 11, 2024

Trier , Germany

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
690
Total Downloads

Downloads (Last 12 months)121
Downloads (Last 6 weeks)18

Reflects downloads up to 01 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Effing RHinz M(2024)Are Children Ready for the Metaverse? The Minefield of Virtual Participation in Digital Social Spaces with Harmful Content and BehaviorElectronic Participation10.1007/978-3-031-70804-6_1(1-15)Online publication date: 25-Aug-2024
https://doi.org/10.1007/978-3-031-70804-6_1
Norval CCloete RSingh J(2023)Navigating the Audit Landscape: A Framework for Developing Transparent and Auditable XRProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3594090(1418-1431)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3594090
Venugopal JSubramanian APeatchimuthu J(2023)The realm of metaverse: A surveyComputer Animation and Virtual Worlds10.1002/cav.215034:5Online publication date: 2-Mar-2023
https://doi.org/10.1002/cav.2150
Cloete RNorval CSingh J(2021)Auditable Augmented/Mixed/Virtual RealityProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34950015:4(1-24)Online publication date: 30-Dec-2021
https://dl.acm.org/doi/10.1145/3495001
Norval CCloete RMarkovic MNaja ICornelius K(2021)Workshop on Reviewable and Auditable Pervasive Systems (WRAPS)Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers10.1145/3460418.3479265(261-263)Online publication date: 21-Sep-2021
https://dl.acm.org/doi/10.1145/3460418.3479265
Cobbe JLee MSingh J(2021)Reviewable Automated Decision-MakingProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency10.1145/3442188.3445921(598-609)Online publication date: 3-Mar-2021
https://dl.acm.org/doi/10.1145/3442188.3445921

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents