Abstract
Biomedical instance segmentation is vulnerable to complicated instance morphology, resulting in over-merge and over-segmentation. Recent advanced methods apply convolutional neural networks to predict pixel embeddings to overcome this problem. However, these methods suffer from heavy computational burdens and massive storage. In this paper, we present the first knowledge distillation method tailored for biomedical instance segmentation to transfer the knowledge from a cumbersome teacher network to a lightweight student one. Different from existing distillation methods on other tasks, we consider three kinds of essential knowledge of the instance segmentation task, i.e., instance-level features, instance relationships in the feature space and pixel-level instance boundaries. Specifically, we devise two distillation schemes: (i) instance graph distillation that transfers the knowledge of instance-level features and instance relationships by the instance graphs built from embeddings of the teacher-student pair, respectively, and (ii) pixel affinity distillation that converts pixel embeddings into pixel affinities and explicitly transfers the structured knowledge of instance boundaries encoded in affinities. Experimental results on a 3D electron microscopy dataset (CREMI) and a 2D plant phenotype dataset (CVPPP) demonstrate that the student models trained through our distillation method use fewer than 1% parameters and less than 10% inference time while achieving promising performance compared with corresponding teacher models. Code is available at https://github.com/liuxy1103/BISKD.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Anas, E.M.A., et al.: Clinical target-volume delineation in prostate brachytherapy using residual neural networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 365–373. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_42
Avelar, P.H., Tavares, A.R., da Silveira, T.L., Jung, C.R., Lamb, L.C.: Superpixel image classification with graph attention networks. In: 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 203–209. IEEE (2020)
Beier, T., et al.: Multicut brings automated neurite segmentation closer to human performance. Nat. Methods 14(2), 101–102 (2017)
Chen, H., Qi, X., Yu, L., Heng, P.A.: DCAN: deep contour-aware networks for accurate gland segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2487–2496 (2016)
Chen, L., Strauch, M., Merhof, D.: Instance segmentation of biomedical images with an object-aware embedding learned with local constraints. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 451–459. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_50
Chen, P., Liu, S., Zhao, H., Jia, J.: Distilling knowledge via knowledge review. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5008–5017 (2021)
CREMI: Miccal challenge on circuit reconstruction from electron microscopy images (2016). https://cremi.org/
Dong, M., et al.: Instance segmentation from volumetric biomedical images without voxel-wise labeling. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 83–91. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_10
Funke, J., et al.: Large scale image segmentation with structured loss based deep learning for connectome reconstruction. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1669–1680 (2018)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)
Hinton, G., Vinyals, O., Dean, J., et al.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 2(7) (2015)
Hu, B., Zhou, S., Xiong, Z., Wu, F.: Cross-resolution distillation for efficient 3D medical image registration. IEEE Trans. Circuits Syst. Video Technol. (2022)
Huang, W., et al.: Semi-supervised neuron segmentation via reinforced consistency learning. IEEE Trans. Med. Imaging (2022)
Huang, W., Deng, S., Chen, C., Fu, X., Xiong, Z.: Learning to model pixel-embedded affinity for homogeneous instance segmentation. In: Proceedings of AAAI Conference on Artificial Intelligence (2022)
Ke, T.W., Hwang, J.J., Liu, Z., Yu, S.X.: Adaptive affinity fields for semantic segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 587–602 (2018)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Kulikov, V., Lempitsky, V.: Instance segmentation of biological images using harmonic embeddings. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3843–3851 (2020)
Lee, K., Lu, R., Luther, K., Seung, H.S.: Learning and segmenting dense voxel embeddings for 3D neuron reconstruction. IEEE Trans. Med. Imaging 40(12), 3801–3811 (2021)
Li, M., Chen, C., Liu, X., Huang, W., Zhang, Y., Xiong, Z.: Advanced deep networks for 3D mitochondria instance segmentation. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–5. IEEE (2022)
Liu, X., Huang, W., Zhang, Y., Xiong, Z.: Biological instance segmentation with a superpixel-guided graph. In: IJCAI (2022)
Liu, X., et al.: Learning neuron stitching for connectomics. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12908, pp. 435–444. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87237-3_42
Meilă, M.: Comparing clusterings by the variation of information. In: Schölkopf, B., Warmuth, M.K. (eds.) COLT-Kernel 2003. LNCS (LNAI), vol. 2777, pp. 173–187. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-45167-9_14
Minervini, M., Fischbach, A., Scharr, H., Tsaftaris, S.A.: Finely-grained annotated datasets for image-based plant phenotyping. Pattern Recogn. Lett. 81, 80–89 (2016)
Payer, C., Štern, D., Neff, T., Bischof, H., Urschler, M.: Instance segmentation and tracking with cosine embeddings and recurrent hourglass networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_1
Qin, D., et al.: Efficient medical image segmentation based on knowledge distillation. IEEE Trans. Med. Imaging 40(12), 3820–3831 (2021)
Rand, W.M.: Objective criteria for the evaluation of clustering methods. J. Am. Stat. Assoc. 66(336), 846–850 (1971)
Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., Bengio, Y.: Fitnets: hints for thin deep nets. arXiv preprint arXiv:1412.6550 (2014)
Scharr, H., Minervini, M., Fischbach, A., Tsaftaris, S.A.: Annotated image datasets of rosette plants. In: ECCV (2014)
Tung, F., Mori, G.: Similarity-preserving knowledge distillation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1365–1374 (2019)
Wolf, S., et al.: The mutex watershed and its objective: efficient, parameter-free graph partitioning. IEEE Trans. Pattern Anal. Mach. Intell. (2020)
Xiao, Z., Fu, X., Huang, J., Cheng, Z., Xiong, Z.: Space-time distillation for video super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2113–2122 (2021)
Zagoruyko, S., Komodakis, N.: Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer. arXiv preprint arXiv:1612.03928 (2016)
Acknowledgement
This work was supported in part by the National Key R &D Program of China under Grant 2017YFA0700800, the National Natural Science Foundation of China under Grant 62021001, the University Synergy Innovation Program of Anhui Province No. GXXT-2019-025, and Anhui Provincial Natural Science Foundation under grant No. 1908085QF256.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, X., Hu, B., Huang, W., Zhang, Y., Xiong, Z. (2022). Efficient Biomedical Instance Segmentation via Knowledge Distillation. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13434. Springer, Cham. https://doi.org/10.1007/978-3-031-16440-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-16440-8_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16439-2
Online ISBN: 978-3-031-16440-8
eBook Packages: Computer ScienceComputer Science (R0)