Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Augmented reality with HoloLens in parotid surgery: how to assess and to improve accuracy

  • Head and Neck
  • Published:
European Archives of Oto-Rhino-Laryngology Aims and scope Submit manuscript

Abstract

Purpose

Augmented reality improves planning and execution of surgical procedures. The aim of this study was to evaluate the feasibility of a 3D augmented reality hologram in live parotic surgery. Another goal was to develop an accuracy measuring instrument and to determine the accuracy of the system.

Methods

We created a software to build and manually align 2D and 3D augmented reality models generated from MRI data onto the patient during surgery using the HoloLens® 1 (Microsoft Corporation, Redmond, USA). To assess the accuracy of the system, we developed a specific measuring tool applying a standard electromagnetic navigation device (Fiagon GmbH, Hennigsdorf, Germany).

Results

The accuracy of our system was measured during real surgical procedures. Training of the experimenters and the use of fiducial markers significantly reduced the accuracy of holographic system (p = 0.0166 and p = 0.0132). Precision of the developed measuring system was very high with a mean error of the basic system of 1.3 mm. Feedback evaluation demonstrated 86% of participants agreed or strongly agreed that the HoloLens will play a role in surgical education. Furthermore, 80% of participants agreed or strongly agreed that the HoloLens is feasible to be introduced in clinical routine and will play a role within surgery in the future.

Conclusion

The use of fiducial markers and repeated training reduces the positional error between the hologram and the real structures. The developed measuring device under the use of the Fiagon navigation system is suitable to measure accuracies of holographic augmented reality images of the HoloLens.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Porpiglia F, Fiori C, Checcucci E, Amparore D, Bertolo R (2018) Augmented reality robot-assisted radical prostatectomy: preliminary experience. Urology 115:184. https://doi.org/10.1016/j.urology.2018.01.028

    Article  PubMed  Google Scholar 

  2. Elmi-Terander A, Nachabe R, Skulason H, Pedersen K, Söderman M, Racadio J, Babic D, Gerdhem P, Edström E (2018) Feasibility and accuracy of thoracolumbar minimally invasive pedicle screw placement with augmented reality navigation technology. Spine (Phila Pa) 43(14):1018–1023. https://doi.org/10.1097/brs.0000000000002502

    Article  Google Scholar 

  3. Bong JH, Song HJ, Oh Y, Park N, Kim H, Park S (2018) Endoscopic navigation system with extended field of view using augmented reality technology. Int J Med Robot. https://doi.org/10.1002/rcs.1886

    Article  PubMed  Google Scholar 

  4. Kersten-Oertel M, Jannin P, Collins DL (2012) DVV: a taxonomy for mixed reality visualization in image guided surgery. IEEE Trans Vis Comput Graph 18(2):332–352

    Article  Google Scholar 

  5. Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE TRANS Inf Syst E77-D(12):1321–1329

    Google Scholar 

  6. Scherl C, Stratemeier J, Hesser J, Schönberg S, Lammert A, Servais J, Männle D, Rotter N (2020) Augmented reality with hololens in parotid tumor surgery: a prospective feasibility study. Laryngo-Rhino-Otologie 99(S02):62. https://doi.org/10.1055/s-0040-1710872

    Article  Google Scholar 

  7. Vassallo R, Rankin A, Chen ECS, Peters TM (2017) Hologram stability evaluation for Microsoft HoloLens. In: Kupinski MA, Nishikawa RM (eds) Proceedings of SPIE. Medical imaging 2017: Image perception, observer performance, and technology assessment, vol 10136. Society of Photo-Optical Instrumentation Engineers (SPIE), Bellingham, Washington, USA. https://doi.org/10.1117/12.2255831

  8. Perkins SL, Lin MA, Srinivasan S, Wheeler AJ, Hargreaves BA, Daniel BL (2017) A mixed-reality system for breast surgical planning. In: 2017 IEEE International symposium on mixed and augmented reality (ISMAR-Adjunct), Nantes, France, 9-13 October 2017. IEEE, New York City, pp 269–274. https://doi.org/10.1109/ISMAR-Adjunct.2017.92

  9. Unity Technologies. https://unity.com. Accessed 19 July 2020

  10. A multi-platform, free and open source software package for visualization and medical image computing. https://www.slicer.org. Accessed 19 July 2020

  11. Kikinis R, Pieper SD, Vosburgh KG (2014) 3D Slicer: a Platform for subject-specific image analysis, visualization, and clinical support. In: Jolesz FA (ed) Intraoperative imaging and image-guided therapy. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7657-3_19

    Chapter  Google Scholar 

  12. Microsoft Visual Studio. https://visualstudio.microsoft.com. Accessed 19 July 2020

  13. Kristin J, Mucha D, Schipper J, Klenzner T (2012) Registration strategies for the application of the navigation system Fiagon at the lateral scull base. Laryngorhinootologie 91(5):306–310

    Article  CAS  Google Scholar 

  14. Yoon JW, Chen RE, Kim EJ, Akinduro OO, Kerezoudis P, Han PK, Si P, Freeman WD, Diaz RJ, Komotar RJ (2018) Augmented reality for the surgeon: systematic review. Int J Med Robot Comput Assist Surgery 14(4):e1914

    Article  Google Scholar 

  15. Kersten-Oertel M, Jannin P, Collins DL (2013) The state of the art of visualization in mixed reality image guided surgery. Comput Med Imaging Graph 37(2):98–112

    Article  Google Scholar 

  16. Qian L, Barthel A, Johnson A, Osgood G, Kazanzides P, Navab N, Fuerst B (2017) Comparison of optical see-through head-mounted displays for surgical interventions with object-anchored 2D-display. Int J CARS 12:901–910. https://doi.org/10.1007/s11548-017-1564-y

    Article  Google Scholar 

  17. Barber SR, Jain S, Son Y-J, Chang EH (2018) Virtual functional endoscopic sinus surgery simulation with 3D-printed models for mixed-reality nasal endoscopy. Otolaryngol-Head Neck Surg 159(5):933–937

    Article  Google Scholar 

  18. Li L, Yang J, Chu Y, Wu W, Xue J, Liang P, Chen L (2016) A novel augmented reality navigation system for endoscopic sinus and skull base surgery: a feasibility study. PLoS ONE 11(1):e0146996

    Article  Google Scholar 

  19. van Doormaal TP, van Doormaal JA, Mensink T (2019) Clinical accuracy of holographic navigation using point-based registration on augmented-reality glasses. Oper Neurosurg 17(6):588–593

    Article  Google Scholar 

  20. Grimpampi E, Camomilla V, Cereatti A, De Leva P, Cappozzo A (2013) Metrics for describing soft-tissue artefact and its effect on pose, size, and shape of marker clusters. IEEE Trans Biomed Eng 61(2):362–367

    Article  Google Scholar 

  21. Pepe A, Trotta GF, Mohr-Ziak P, Gsaxner C, Wallner J, Bevilacqua V, Egger J (2019) A marker-less registration approach for mixed reality-aided maxillofacial surgery: a pilot evaluation. J Digit Imaging 32(6):1008–1018

    Article  Google Scholar 

  22. Barsom EZ, Graafland M, Schijven MP (2016) Systematic review on the effectiveness of augmented reality applications in medical training. Surg Endosc 30(10):4174–4183

    Article  CAS  Google Scholar 

  23. Al Janabi HF, Aydin A, Palaneer S, Macchione N, Al-Jabir A, Khan MS, Dasgupta P, Ahmed K (2020) Effectiveness of the HoloLens mixed-reality headset in minimally invasive surgery: a simulation-based feasibility study. Surg Endosc 34(3):1143–1149

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank the Fraunhofer Institute in Mannheim and DFC-SYSTEMS GmbH, Munich for making the HoloLens available.

Funding

The authors did not receive any funding.

Author information

Authors and Affiliations

  1. Department of Otorhinolaryngology, Head and Neck Surgery, University Medical Center Mannheim, Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany

    Claudia Scherl, Nicole Rotter, Lena Huber, Andre Dias, Oliver Hoffmann, Angela Schell, Anne Lammert, Annette Affolter & David Männle

  2. Institute of Experimental Radiation Oncology, University Medical Center Mannheim, Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany

    Johanna Stratemeier, Celine Karle & Jürgen Hesser

  3. Department of Clinical Radiology and Nuclear Medicine, University Medical Center Mannheim, Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany

    Philipp Riffel & Stefan O. Schoenberg

Authors
  1. Claudia Scherl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Johanna Stratemeier
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Celine Karle
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nicole Rotter
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jürgen Hesser
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lena Huber
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Andre Dias
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Oliver Hoffmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Philipp Riffel
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Stefan O. Schoenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Angela Schell
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Anne Lammert
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Annette Affolter
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. David Männle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Claudia Scherl.

Ethics declarations

Conflicts of interest

The authors have no conflicts of interest to declare.

Ethics approval

The research was conducted ethically in accordance with the World Medical Association Declaration of Helsinki. Subjects have given their written informed consent and the study protocol was approved by the institute’s committee of ethics (#2019-739 N). The setup of the current study did not impose any additional risks to patients or surgeons, because they were not subject to any specific procedure, nor were they required to follow any new rules of behavior. Diagnostics and surgery were performed without any deviation from the standard procedure.

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Additional file1 (MP4 134837 kb)

Additional file2 (MP4 53612 kb)

Additional file3 (MP4 46071 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Scherl, C., Stratemeier, J., Karle, C. et al. Augmented reality with HoloLens in parotid surgery: how to assess and to improve accuracy. Eur Arch Otorhinolaryngol 278, 2473–2483 (2021). https://doi.org/10.1007/s00405-020-06351-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00405-020-06351-7

Keywords

Search

Navigation