Abstract
Medical image segmentation is crucial in the field of medical imaging, assisting healthcare professionals in analyzing images and improving diagnostic performance. Recent advancements in Transformer-based networks, which utilize self-attention mechanism, have proven their effectiveness in various medical problems, including medical imaging. However, existing self-attention mechanism in Transformers only captures pairwise correlations among image patches, neglecting non-pairwise correlations that are essential for performance enhancement. On the other hand, recently, graph-based networks have emerged to capture both pairwise and non-pairwise correlations effectively. Inspired by recent Hypergraph Neural Network (HGNN), we propose a novel hypergraph-based network for medical image segmentation. Our contribution lies in formulating novel and efficient HGNN methods for constructing Hyperedges. To effectively aggregate multiple patches with similar attributes at both feature and local levels, we introduce an improved adaptive technique leveraging the K-Nearest Neighbors (KNN) algorithm to enhance the hypergraph construction process. Additionally, we generalize the concept of Convolutional Neural Networks (CNNs) to hypergraphs. Our method achieves state-of-the-art results on two publicly available segmentation datasets, and visualization results further validate its effectiveness. Our code is released on Github: https://github.com/11yxk/AHGNN.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cao, H., et al.: Swin-unet: Unet-like pure transformer for medical image segmentation. arXiv preprint arXiv:2105.05537 (2021)
Chen, J., et al.: Transunet: transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021)
Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
Feng, Y., You, H., Zhang, Z., Ji, R., Gao, Y.: Hypergraph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3558–3565 (2019)
Fu, S., et al.: Domain adaptive relational reasoning for 3D multi-organ segmentation. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 656–666. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_64
Han, K., Wang, Y., Guo, J., Tang, Y., Wu, E.: Vision GNN: an image is worth graph of nodes. In: Advances in Neural Information Processing Systems, vol. 35, pp. 8291–8303 (2022)
Han, Y., Wang, P., Kundu, S., Ding, Y., Wang, Z.: Vision HGNN: an image is more than a graph of nodes. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 19878–19888 (2023)
Huang, H., et al.: Unet 3+: a full-scale connected unet for medical image segmentation. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1055–1059. IEEE (2020)
Huang, H., et al.: Scaleformer: revisiting the transformer-based backbones from a scale-wise perspective for medical image segmentation. arXiv preprint arXiv:2207.14552 (2022)
Huang, X., Deng, Z., Li, D., Yuan, X.: Missformer: an effective medical image segmentation transformer. arXiv preprint arXiv:2109.07162 (2021)
Khan, S., Naseer, M., Hayat, M., Zamir, S.W., Khan, F.S., Shah, M.: Transformers in vision: a survey. ACM Comput. Surv. (CSUR) 54(10s), 1–41 (2022)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Liang, Z., Zhao, K., Liang, G., Li, S., Wu, Y., Zhou, Y.: Maxformer: enhanced transformer for medical image segmentation with multi-attention and multi-scale features fusion. Knowl.-Based Syst. 280, 110987 (2023)
Liu, Y., Wang, H., Chen, Z., Huangliang, K., Zhang, H.: Transunet+: redesigning the skip connection to enhance features in medical image segmentation. Knowl.-Based Syst. 256, 109859 (2022)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Lostar, M., Rekik, I.: Deep hypergraph u-net for brain graph embedding and classification. arXiv preprint arXiv:2008.13118 (2020)
Milletari, F., Navab, N., Ahmadi, S.A.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571. IEEE (2016)
Peng, J., et al.: Make u-net greater: an easy-to-embed approach to improve segmentation performance using hypergraph. Comput. Syst. Sci. Eng. 42(1), 319–333 (2022)
Rahman, M.M., Marculescu, R.: Medical image segmentation via cascaded attention decoding. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 6222–6231 (2023)
Rahman, M.M., Marculescu, R.: Multi-scale hierarchical vision transformer with cascaded attention decoding for medical image segmentation. In: Medical Imaging with Deep Learning, pp. 1526–1544. PMLR (2024)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Wang, H., et al.: Mixed transformer u-net for medical image segmentation. In: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2390–2394. IEEE (2022)
Yan, X., Tang, H., Sun, S., Ma, H., Kong, D., Xie, X.: After-unet: axial fusion transformer unet for medical image segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3971–3981 (2022)
Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans. Med. Imaging 39(6), 1856–1867 (2019)
Acknowledgments
This work was supported in part by Ritsumeikan Advanced Research Academy (RARA) Program and the Grant in Aid for Scientific Research from the Japanese Ministry for Education, Science, Culture and Sports (MEXT) under the Grant Nos. 20KK0234, 21H03470, and 20K21821, and in part by Zhejiang Provincial Natural Science Foundation of China (No. LZ22F020012).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The manuscript is approved for publication by all authors. All authors declare no conflicts of interest regarding the submission and publication.
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chai, S. et al. (2024). A Novel Adaptive Hypergraph Neural Network for Enhancing Medical Image Segmentation. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15009. Springer, Cham. https://doi.org/10.1007/978-3-031-72114-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-72114-4_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72113-7
Online ISBN: 978-3-031-72114-4
eBook Packages: Computer ScienceComputer Science (R0)