Abstract
Pituitary tumours are in an anatomically dense region of the body, and often distort or encase the surrounding critical structures. This, in combination with anatomical variations and limitations imposed by endoscope technology, makes intra-operative identification and protection of these structures challenging. Advances in machine learning have allowed for the opportunity to automatically identifying these anatomical structures within operative videos. However, to the best of the authors’ knowledge, this remains an unaddressed problem in the sellar phase of endoscopic pituitary surgery. In this paper, PAINet (Pituitary Anatomy Identification Network), a multi-task network capable of identifying the ten critical anatomical structures, is proposed. PAINet jointly learns: (1) the semantic segmentation of the two most prominent, largest, and frequently occurring structures (sella and clival recess); and (2) the centroid detection of the remaining eight less prominent, smaller, and less frequently occurring structures. PAINet utilises an EfficientNetB3 encoder and a U-Net++ decoder with a convolution layer for segmentation and pooling layer for detection. A dataset of 64-videos (635 images) were recorded, and annotated for anatomical structures through multi-round expert consensus. Implementing 5-fold cross-validation, PAINet achieved 66.1% and 54.1% IoU for sella and clival recess semantic segmentation respectively, and 53.2% MPCK-20% for centroid detection of the remaining eight structures, improving on single-task performances. This therefore demonstrates automated identification of anatomical critical structures in the sellar phase of endoscopic pituitary surgery is possible.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Danks, R.P., et al.: Automating periodontal bone loss measurement via dental landmark localisation. Int. J. Comput. Assist. Radiol. Surg. 16(7), 1189–1199 (2021). https://doi.org/10.1007/s11548-021-02431-z
Gaggion, N., Mansilla, L., Mosquera, C., Milone, D.H., Ferrante, E.: Improving anatomical plausibility in medical image segmentation via hybrid graph neural networks: applications to chest x-ray analysis. IEEE Trans. Med. Imaging 42(2), 546–556 (2023). https://doi.org/10.1109/tmi.2022.3224660
Gu, R., et al.: Contrastive semi-supervised learning for domain adaptive segmentation across similar anatomical structures. IEEE Trans. Med. Imaging 42(1), 245–256 (2023). https://doi.org/10.1109/tmi.2022.3209798
Hao, S., Zhou, Y., Guo, Y.: A brief survey on semantic segmentation with deep learning. Neurocomputing 406, 302–321 (2020). https://doi.org/10.1016/j.neucom.2019.11.118
Jin, Y., Yu, Y., Chen, C., Zhao, Z., Heng, P.A., Stoyanov, D.: Exploring intra- and inter-video relation for surgical semantic scene segmentation. IEEE Trans. Med. Imaging 41(11), 2991–3002 (2022). https://doi.org/10.1109/tmi.2022.3177077
Liu, L., Wolterink, J.M., Brune, C., Veldhuis, R.N.J.: Anatomy-aided deep learning for medical image segmentation: a review. Phys. Med. Biol. 66(11), 11TR01 (2021). https://doi.org/10.1088/1361-6560/abfbf4
Madani, A., et al.: Artificial intelligence for intraoperative guidance using semantic segmentation to identify surgical anatomy during laparoscopic cholecystectomy. Ann. Surg. 276(2), 363–369 (2020). https://doi.org/10.1097/sla.0000000000004594
Maier-Hein, L., Reinke, A., Godau, P., et al.: Metrics reloaded: pitfalls and recommendations for image analysis validation (2022). https://doi.org/10.48550/arxiv.2206.01653
Marcus, H.J., et al.: Pituitary society expert Delphi consensus: operative workflow in endoscopic transsphenoidal pituitary adenoma resection. Pituitary 24(6), 839–853 (2021). https://doi.org/10.1007/s11102-021-01162-3
Marullo, G., Tanzi, L., Ulrich, L., Porpiglia, F., Vezzetti, E.: A multi-task convolutional neural network for semantic segmentation and event detection in laparoscopic surgery. J. Personal. Med. 13(3), 413 (2023). https://doi.org/10.3390/jpm13030413
Patel, C.R., Fernandez-Miranda, J.C., Wang, W.H., Wang, E.W.: Skull base anatomy. Otolaryngol. Clin. North Am. 49(1), 9–20 (2016). https://doi.org/10.1016/j.otc.2015.09.001
Staartjes, V.E., Volokitin, A., Regli, L., Konukoglu, E., Serra, C.: Machine vision for real-time intraoperative anatomic guidance: a proof-of-concept study in endoscopic pituitary surgery. Oper. Neurosurg. 21(4), 242–247 (2021). https://doi.org/10.1093/ons/opab187
Tan, M., Le, Q.V.: EfficientNet: rethinking model scaling for convolutional neural networks. arXiv (2019). https://doi.org/10.48550/ARXIV.1905.11946
Wang, P., Peng, J., Pedersoli, M., Zhou, Y., Zhang, C., Desrosiers, C.: CAT: constrained adversarial training for anatomically-plausible semi-supervised segmentation. IEEE Trans. Med. Imaging, 1 (2023). https://doi.org/10.1109/tmi.2023.3243069
Zhang, Y., Yang, Q.: An overview of multi-task learning. Natl. Sci. Rev. 5(1), 30–43 (2017). https://doi.org/10.1093/nsr/nwx105
Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: UNet++: a nested U-net architecture for medical image segmentation. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_1
Acknowledgements
This research was funded in whole, or in part, by the Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS) [203145/Z/16/Z]; the Engineering and Physical Sciences Research Council (EPSRC) [EP/P027938/1, EP/R004080/1, EP/P012841/1, EP/W00805X/1]; and the Royal Academy of Engineering Chair in Emerging Technologies Scheme. AD is supported by EPSRC [EP/S021612/1]. HJM is supported by WEISS [NS/A000050/1] and by the National Institute for Health and Care Research (NIHR) Biomedical Research Centre at University College London (UCL). DZK and JGH are supported by the NIHR Academic Clinical Fellowship. DZK is supported by the Cancer Research UK (CRUK) Predoctoral Fellowship. With thanks to Digital Surgery Ltd, a Medtronic company, for access to Touch SurgeryTM Enterprise for both video recording and storage.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Das, A. et al. (2023). A Multi-task Network for Anatomy Identification in Endoscopic Pituitary Surgery. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14228. Springer, Cham. https://doi.org/10.1007/978-3-031-43996-4_45
Download citation
DOI: https://doi.org/10.1007/978-3-031-43996-4_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43995-7
Online ISBN: 978-3-031-43996-4
eBook Packages: Computer ScienceComputer Science (R0)