Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

A Holographic Augmented Reality Interface for Visualizing of MRI Data and Planning of Neurosurgical Procedures

  • Original Paper
  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

The recent introduction of wireless head-mounted displays (HMD) promises to enhance 3D image visualization by immersing the user into 3D morphology. This work introduces a prototype holographic augmented reality (HAR) interface for the 3D visualization of magnetic resonance imaging (MRI) data for the purpose of planning neurosurgical procedures. The computational platform generates a HAR scene that fuses pre-operative MRI sets, segmented anatomical structures, and a tubular tool for planning an access path to the targeted pathology. The operator can manipulate the presented images and segmented structures and perform path-planning using voice and gestures. On-the-fly, the software uses defined forbidden-regions to prevent the operator from harming vital structures. In silico studies using the platform with a HoloLens HMD assessed its functionality and the computational load and memory for different tasks. A preliminary qualitative evaluation revealed that holographic visualization of high-resolution 3D MRI data offers an intuitive and interactive perspective of the complex brain vasculature and anatomical structures. This initial work suggests that immersive experiences may be an unparalleled tool for planning neurosurgical procedures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Kersten-Oertel M, Gerard IJ, Drouin S, et al: “Towards Augmented Reality Guided Craniotomy Planning in Tumour Resections,” Springer, Cham, pp. 163–174, 2016.

    Google Scholar 

  2. Cutolo F, Freschi C, Mascioli S, et al: “Robust and Accurate Algorithm for Wearable Stereoscopic Augmented Reality with Three Indistinguishable Markers,” Electronics 5(4):59, 2016.

    Article  Google Scholar 

  3. Kim KH, “The Potential Application of Virtual, Augmented, and Mixed Reality in Neurourology,” Int Neurourol J, 20(3):169–170, 2016.

    Article  Google Scholar 

  4. Kuhlemann I, Kleemann I, Jauer P, et al: “Towards X-ray Free Endovascular Interventions – Using HoloLens for On-line Holographic Visualisation,” Healthc Technol Lett, 4(5):184–187, 2017.

    Article  Google Scholar 

  5. Chen AD and Lin SJ, “Discussion: Mixed Reality with HoloLens: Where Virtual Reality Meets Augmented Reality in the Operating Room.,” Plast Reconstr Surg, 140(5):1071–1072, 2017.

    Article  CAS  Google Scholar 

  6. Tepper OM, Rudy HL, Lefkowitz A, et al: “Mixed Reality with HoloLens: Where Virtual Reality Meets Augmented Reality in the Operating Room.,” Plast Reconstr Surg, 140(5):1066–1070, 2017.

    Article  CAS  Google Scholar 

  7. Qian L, Barthel A, Johnson A, et al: “Comparison of Optical See-through Head-mounted Displays for Surgical Interventions with Object-anchored 2D-display,” Int J Comput Assist Radiol Surg, 12(6):901–910, 2017.

    Article  Google Scholar 

  8. Nabavi A, Gering DT, Kacher DF, et al: “Surgical Navigation in the Open MRI,” in Acta Neurochirurgica, Supplement, 85:121–125, 2003.

    Article  CAS  Google Scholar 

  9. Kral F, Mehrle A, Kikinis R, et al: “Using CAVE Technology for Educating and Training Residents,” Otolaryngol Neck Surg, 131(2):P210–P210, 2004.

    Article  Google Scholar 

  10. Fedorov A, Beichel R, Kalpathy-Cramer J, et al: “3D Slicer as an Image Computing Platform for the Quantitative Imaging Network,” Magn Reson Imaging, 30(9):1323–1341, 2012.

    Article  Google Scholar 

  11. Jolesz FA, Nabavi A, Kikinis R, et al: “Integration of Interventional MRI with Computer-assisted Surgery,” J Magn Reson Imaging, 13(1):69–77, 2001.

    Article  CAS  Google Scholar 

  12. Black PM, Moriarty T, Alexander E, et al: “Development and Implementation of Intraoperative Magnetic Resonance Imaging and Its Neurosurgical Applications,” Neurosurgery, 41(4):831–845, 1997.

    Article  CAS  Google Scholar 

  13. Guttman MA, Lederman RJ, Sorger JM, et al: “Real-time Volume Rendered MRI for Interventional Guidance,” J Cardiovasc Magn Reson, 4(4):431–442, 2002.

    Article  Google Scholar 

  14. George AK, Derbyshire JA, Saybasili H, et al: “Visualization of Active Devices and Automatic Slice Repositioning (‘SnapTo’) for MRI-guided Interventions,” Magn Reson Med, 63(4):1070–1079, 2010.

    Article  Google Scholar 

  15. McVeigh ER, Guttman MA, Kellman P, et al., “Real-time, Interactive MRI for Cardiovascular Interventions,” in Academic Radiology, 12(9):1121–1127, 2005.

    Article  Google Scholar 

  16. Wacker FK, Elgort D, Hillenbrand CM, et al: “The Catheter-Driven MRI Scanner: A New Approach to Intravascular Catheter Tracking and Imaging-Parameter Adjustment for Interventional MRI,” Am J Roentgenol, 183(2):391–395, 2004.

    Article  Google Scholar 

  17. Christoforou E, Akbudak E, Ozcan A, et al: “Performance of Interventions with Manipulator-driven Real-time MR Guidance: Implementation and Initial in Vitro Tests,” Magn Reson Imaging, 25(1):69–77, 2007.

    Article  Google Scholar 

  18. Velazco Garcia JD, Navkar NV, Gui D, et al: A platform integrating acquisition, reconstruction, visualization, and manipulator control modules for MRI-guided interventions, J Digit Imaging, 32(3):420–432, 2019.

  19. Guttman MA and McVeigh ER, “Techniques for Fast Stereoscopic MRI,” Magn Reson Med, 46(2):317–323, 2001.

    Article  CAS  Google Scholar 

  20. Gobbetti E, Pili P, Zorcolo A, et al: “Interactive Virtual Angioscopy,” in Proceedings Visualization ’98(Cat. No.98CB36276), 435–438, 1998.

  21. Muntener M, Patriciu A, Petrisor D, et al: “Magnetic Resonance Imaging Compatible Robotic System for Fully Automated Brachytherapy Seed Placement,” Urology, 68(6):1313–1317, 2006.

    Article  Google Scholar 

  22. Gomez-Rodriguez M, Peters J, Hill J, et al: Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery, J Neural Eng, 8(3), 2011.

  23. Morales Mojica CM, Navkar NV, Tsekos NV, et al: Holographic interface for three-dimensional visualization of MRI on HoloLens: a prototype platform for MRI guided neurosurgeries,” in Proceedings - 2017 IEEE 17th International Conference on Bioinformatics and Bioengineering, BIBE 2017.

  24. Microsoft, “The leader in mixed reality technology | hololens,” Microsoft, 2017. [Online]. Available: https://www.microsoft.com/en-us/hololens. Accessed 16 Nov 2019.

  25. Navkar NV, Deng Z, Shah DJ, et al: “A Framework for Integrating Real-time MRI with Robot Control: Application to Simulated Transapical Cardiac Interventions,” IEEE Trans Biomed Eng, 60(4):1023–1033, 2013.

    Article  Google Scholar 

  26. Velazco-Garcia JD, Leiss EL, Karkoub M, et al: Preliminary evaluation of robotic transrectal biopsy system on an interventional planning software, in Proceedings - 2019 IEEE 19th International Conference on Bioinformatics and Bioengineering, BIBE, 357–362, 2019.

  27. Morales Mojica CM, Velazco-Garcia JD, Navkar NV, et al: A prototype holographic augmented reality interface for image-guided prostate cancer interventions, Proc Eurographics Work Vis Comput Biol Med, 17–21, 2018.

  28. Morales Mojica CM, Velazco-Garcia JD, Navkar NV, et al: A prototype holographic augmented reality interface for image-guided prostate cancer interventions, Proc Eurographics Work Vis Comput Biol Med, 17–21, 2018.

    Article  Google Scholar 

  29. Velazco‐Garcia JD, Navkar NV, Balakrishnan S, et al: End-user evaluation of software-generated intervention planning environment for transrectal magnetic resonance-guided prostate biopsies, Int J Med Robot Comput Assist Surg, 2020.

  30. Velazco-Garcia JD, Navkar NV, Balakrishnan S, et al: Evaluation of interventional planning software features for MR-guided transrectal prostate biopsies, in Proceedings - 2020 IEEE 20th International Conference on Bioinformatics and Bioengineering, BIBE, 2020.

  31. Molina G, Velazco-Garcia JD, Shah D, et al: Automated segmentation and 4D reconstruction of the heart left ventricle from CINE MRI, in Proceedings - 2019 IEEE 19th International Conference on Bioinformatics and Bioengineering, BIBE, 2019.

  32. Kensicher T, Leclerc J, Biediger D, et al: Towards MRI-guided and actuated tetherless milli-robots: preoperative planning and modeling of control, in IEEE International Conference on Intelligent Robots and Systems, 2017, vol. 2017-Septe, pp. 6440–6447.

  33. Pohle R and Toennies KD, Segmentation of medical images using adaptive region growing, in Medical Imaging 2001: Image Processing, 4322:1337–1346, 2001.

  34. Gudbjartsson H and Patz S, “The Rician Distribution of Noisy MRI Data,” Magn Reson Med, 34(6):910–914, 1995.

    Article  CAS  Google Scholar 

  35. “JSON,” 2019. [Online]. Available: http://www.json.org/. Accessed 16 Nov 2019.

  36. Unity Technologies, Unity real-time development platform | 3D, 2D VR and AR visualizations, 2019. [Online]. Available: https://unity.com/. Accessed 24 Sep 2019.

  37. Navkar NV, Deng Z, Shah DJ, et al: Visual and force-feedback guidance for robot-assisted interventions in the beating heart with real-time MRI, in Proceedings - IEEE International Conference on Robotics and Automation, 689–694, 2012.

  38. Rincon-Nigro M, Navkar NV, Tsekos NV, et al: “GPU-accelerated Interactive Visualization and Planning of Neurosurgical Interventions,” IEEE Comput Graph Appl, 34(1)22–31, 2014.

    Article  Google Scholar 

  39. Microsoft, HoloLens environment considerations | Microsoft docs. [Online]. Available: https://docs.microsoft.com/en-us/hololens/hololens-environment-considerations. Accessed 25 Nov 2020.

Download references

Funding

This work was supported by the National Science Foundation award CNS-1646566, DGE-1746046, and DGE-1433817. NVT and IS also received in part support by the Stavros Niarchos Foundation that is administered by the Institute of International Education (IIE). All opinions, findings, conclusions, or recommendations expressed in this work are those of the authors and do not necessarily reflect the views of our sponsors.

Author information

Author notes

  1. Cristina M. Morales Mojica and Jose D. Velazco-Garcia equally contributed on this work.

Authors and Affiliations

  1. MRI Lab, Department of Computer Science, University of Houston, 4800 Calhoun Road PGH 501, Houston, TX, USA

    Cristina M. Morales Mojica, Jose D. Velazco-Garcia, Ernst L. Leiss & Nikolaos V. Tsekos

  2. Medical Physics Laboratory, School of Medicine, National and Kapodistrian University of Athens, Athens, Greece

    Eleftherios P. Pappas & Ioannis Seimenis

  3. School of Medicine, Democritus University of Thrace, Alexandroupolis, Greece

    Theodosios A. Birbilis

  4. Department of Electrical and Computer Engineering, University of Houston, Houston, TX, USA

    Aaron Becker

  5. C.J. Gorter Center for High Field MRI, Leiden University Medical Center, Leiden, Netherlands

    Andrew Webb

Authors
  1. Cristina M. Morales Mojica
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jose D. Velazco-Garcia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eleftherios P. Pappas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Theodosios A. Birbilis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Aaron Becker
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ernst L. Leiss
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Andrew Webb
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Ioannis Seimenis
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Nikolaos V. Tsekos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikolaos V. Tsekos.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (MP4 8714 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Morales Mojica, C.M., Velazco-Garcia, J.D., Pappas, E.P. et al. A Holographic Augmented Reality Interface for Visualizing of MRI Data and Planning of Neurosurgical Procedures. J Digit Imaging 34, 1014–1025 (2021). https://doi.org/10.1007/s10278-020-00412-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-020-00412-3

Keywords

Search

Navigation