Abstract
The dynamical system perspective has been used to build efficient image classification networks and semantic segmentation networks. Furthermore, the Runge–Kutta (RK) methods are powerful tools for building networks from the dynamical systems perspective. Hence, the Runge–Kutta segmentation network (RKSeg) for medical image segmentation was born. Skip connections and multiple scaling are often used in common models but lack mathematical explanations. RKSeg interprets and uses skip connections based on the RK methods. Therefore, RKSeg greatly improves segmentation efficiency. However, it does not explain and use multiple scales from a dynamical system perspective but only inherits the multi-scale scheme of existing models. We compensate for this shortcoming by interpreting and using multiple scales based on the RK methods. In addition, the network structure also limits the excellent image classification networks as the backbones of RKSegs. Therefore, we modify the network structure to support more image classification networks as backbones. As a result, we propose a novel network structure RKSeg+. Our proposed RKSeg+ achieves better segmentation results with fewer parameters than RKSeg. Furthermore, RKSeg+, well configured with few parameters, outperforms state-of-the-art models on six of the ten organ datasets in the Medical Segmentation Decathlon.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
All experimental images are provided by http://medicaldecathlon.com/.
Code availability
The code for RKSeg+ is available at https://github.com/ZhuMai/RKSegPlus.
References
Zhu, M., Fu, C., Wang, X.: Semantic segmentation of medical images based on Runge–Kutta methods. Bioengineering (2023). https://doi.org/10.3390/bioengineering10050506
E, W.: A proposal on machine learning via dynamical systems. Commun. Math. Stat. 5(1), 1–11 (2017). https://doi.org/10.1007/s40304-017-0103-z
Butcher, J.C.: Numerical differential equation methods, pp. 51–135. Wiley, Chichester (2008). https://doi.org/10.1002/9780470753767.ch2
Zhu, M., Chang, B., Fu, C.: Convolutional neural networks combined with Runge–Kutta methods. Neural Comput Appl 35, 1629–1643 (2023). https://doi.org/10.1007/s00521-022-07785-2
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3431–3440 (2015). https://doi.org/10.1109/CVPR.2015.7298965
Butcher, J.C.: 1. Differential and difference equations, pp. 1–49. Wiley, Chichester (2008). https://doi.org/10.1002/9780470753767.ch1
Süli, E., Mayers, D.F.: An introduction to numerical analysis, pp. 351–352. Cambridge University Press, Cambridge (2003). https://doi.org/10.1017/CBO9780511801181
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, Atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018). https://doi.org/10.1109/TPAMI.2017.2699184
Simpson, A.L., Antonelli, M., Bakas, S., Bilello, M., Farahani, K., Van Ginneken, B., Kopp-Schneider, A., Landman, B.A., Litjens, G., Menze, B., et al.: A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXiv:1902.09063 (2019) doi: https://doi.org/10.48550/arXiv.1902.09063
Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Kopp-Schneider, A., Landman, B.A., Litjens, G., Menze, B., Ronneberger, O., Summers, R.M., et al.: The medical segmentation decathlon. Nat. Commun. 13(1), 4128 (2022). https://doi.org/10.1038/s41467-022-30695-9
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Jégou, S., Drozdzal, M., Vazquez, D., Romero, A., Bengio, Y.: The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1175–1183 (2017). https://doi.org/10.1109/CVPRW.2017.156
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269 (2017). https://doi.org/10.1109/CVPR.2017.243
Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: Unet++: a nested u-net architecture for medical image segmentation. In: Stoyanov, D., Taylor, Z., Carneiro, G., Syeda-Mahmood, T., Martel, A., Maier-Hein, L., Tavares, J.M.R.S., Bradley, A., Papa, J.P., Belagiannis, V., Nascimento, J.C., Lu, Z., Conjeti, S., Moradi, M., Greenspan, H., Madabhushi, A. (eds.) Deep learning in medical image analysis and multimodal learning for clinical decision support, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_1
Huang, H., Lin, L., Tong, R., Hu, H., Zhang, Q., Iwamoto, Y., Han, X., Chen, Y.-W., Wu, J.: Unet 3+: a full-scale connected UNET for medical image segmentation. In: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1055–1059 (2020). https://doi.org/10.1109/ICASSP40776.2020.9053405
Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021). https://doi.org/10.1038/s41592-020-01008-z
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv preprint arXiv:1412.7062 (2014) https://doi.org/10.48550/arXiv.1412.7062
Chen, L.-C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587 (2017) https://doi.org/10.48550/arXiv.1706.05587
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision—ECCV 2018, pp. 833–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_49
Funding
This research was supported by the National Natural Science Foundation of China (No. 62032013), and the Fundamental Research Funds for the Central Universities (No. N2324004-12).
Author information
Authors and Affiliations
Contributions
Conceptualization: MZ. Methodology: MZ. Software: MZ. Writing—original draft preparation: MZ. Writing—review and editing: MZ, CF. Funding acquisition: CF, XW. Resources: CF. Supervision: CF.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no competing interests to declare that are relevant to the content of this article.
Ethical approval
The authors used the data provided by MSD. MSD has made all data available online with a permissive copyright license (CC-BY-SA 4.0), allowing for data to be shared, distributed and improved upon. The authors have cited their paper.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Additional information
Communicated by B. Bao.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhu, M., Fu, C. & Wang, X. RKSeg+: make full use of Runge–Kutta methods in medical image segmentation. Multimedia Systems 30, 65 (2024). https://doi.org/10.1007/s00530-024-01263-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00530-024-01263-6