Abstract
Accurate motion tracking of the liver target is crucial in image-guided intervention therapy. Compared with other imaging modalities, ultrasound is appealing choice as it provides accurate and real-time anatomical information surrounding lesions. Besides, compared with 2-dimensional ultrasound (2DUS) image, 3-dimensional ultrasound (3DUS) image shows the spatial structure and real lesion motion pattern in patient so that it is an ideal choice for image-guided intervention. In this work, we develop Siamese Spatial Pyramid Matching Network (SSPMNet) to track anatomical landmark in 3DUS sequences. SSPMNet mainly consists of two parts, namely feature extraction network and decision network. Feature extraction network with fully convolutional neural (FCN) layers is employed to extract the deep feature in 3DUS image. Spatial Pyramid Pooling (SPP) layer is connected to the end of feature extraction network to generate multiple-level and robust anatomical structure features. In decision network, three fully connected layers are used to compute the similarity between features. Moreover, with the prior knowledge of physical movement, we elaborately design a temporal consistency model to reject outliers in tracking results. Proposed algorithm is evaluated on the Challenge of Liver Ultrasound Tracking (CLUST) across 16 3DUS sequences, yielding \( 1.89\; \pm \;1.14 \) mm mean compared with manual annotations. Moreover, extensive ablation study proves that the leading tracking result can benefit from hierarchical feature extraction by SPP. Besides proposed algorithm is not sensitive to sampled sub-volume size. Therefore, proposed algorithm is potential for accurate anatomical landmark tracking in ultrasound-guided intervention.
J. He and C. Shen––The first two authors contribute equally, and the first author is student.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Mageras, G.S., Yorke, E.: Deep inspiration breath hold and respiratory gating strategies for reducing organ motion in radiation treatment. Semin. Radiat. Oncol. 14(1), 65–75 (2004)
Vijayan, S., Klein, S., Hofstad, E.F., Lindseth, F., Ystgaard, B., Langø, T.J.M.: Motion tracking in the liver: validation of a method based on 4D ultrasound using a nonrigid registration technique. Med. Phys. 41(8Patr1), 082903 (2014)
De Senneville, B.D., Mougenot, C., Moonen, C.T.: Real-time adaptive methods for treatment of mobile organs by MRI-controlled high-intensity focused ultrasound. Magn. Reson. Med. Off. J. Int. Soc. Magn. Reson. Med. 57(2), 319–330 (2007)
Dürichen, R., Davenport, L., Bruder, R., Wissel, T., Schweikard, A., Ernst, F.: Evaluation of the potential of multi-modal sensors for respiratory motion prediction and correlation. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5678–5681. IEEE (2013)
Banerjee, J., Klink, C., Peters, E.D., Niessen, W.J., Moelker, A., van Walsum, T.: Fast and robust 3D ultrasound registration–block and game theoretic matching. Med. Image Anal. 20(1), 173–183 (2015)
Royer, L., Krupa, A., Dardenne, G., Le Bras, A., Marchand, E., Marchal, M.: Real-time target tracking of soft tissues in 3D ultrasound images based on robust visual information and mechanical simulation. Med. Image Anal. 35, 582–598 (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2015)
Han, X., Leung, T., Jia, Y., Sukthankar, R., Berg, A.C.: MatchNet: unifying feature and metric learning for patch-based matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3279–3286. IEEE (2015)
Ioffe, S., Szegedy, C.J.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning (ICML), pp. 448–456. ACM (2015)
Yang, H., Shan, C., Kolen, A.F., de With, P.H.: Catheter detection in 3D ultrasound using triplanar-based convolutional neural networks. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 371–375. IEEE (2018)
Lin, M., Chen, Q., Yan, S.J.: Network in network. arXiv preprint arXiv:1312.4400 (2014)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Luca, V., et al.: The 2014 liver ultrasound tracking benchmark. Phys. Med. Biol. 60(14), 5571–5599 (2015)
Chollet, F.: Keras. GitHub (2015). https://github.com/fchollet/keras
Banerjee, J., Klink, C., Vast, E., Niessen, W.J., Moelker, A., van Walsum, T.: A combined tracking and registration approach for tracking anatomical landmarks in 4D ultrasound of the liver. In: MICCAI Workshop: Challenge on Liver Ultrasound Tracking, pp. 36–43 (2015)
Lin, T., Dollar, P., Girshick, R.B., He, K., Hariharan, B., Belongie, S.J.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944. IEEE (2017)
Min, S., Chen, X., Zha, Z.-J., Wu, F., Zhang, Y.: A two-stream mutual attention network for semi-supervised biomedical segmentation with noisy labels. arXiv preprint arXiv:1807.11719 (2019)
Zhao, C., Zhang, P., Zhu, J., Wu, C., Wang, H., Xu, K.: Predicting tongue motion in unlabeled ultrasound videos using convolutional LSTM neural network. arXiv preprint arXiv:1902.06927 (2019)
Acknowledgement
This work is supported in part by Knowledge Innovation Program of Basic Research Projects of Shenzhen under Grant JCYJ20160428182053361, in part by Guangdong Science and Technology Plan under Grant 2017B020210003 and in part by National Natural Science Foundation of China under Grant 81771940, 81427803.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
He, J., Shen, C., Huang, Y., Wu, J. (2019). Siamese Spatial Pyramid Matching Network with Location Prior for Anatomical Landmark Tracking in 3-Dimension Ultrasound Sequence. In: Lin, Z., et al. Pattern Recognition and Computer Vision. PRCV 2019. Lecture Notes in Computer Science(), vol 11858. Springer, Cham. https://doi.org/10.1007/978-3-030-31723-2_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-31723-2_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-31722-5
Online ISBN: 978-3-030-31723-2
eBook Packages: Computer ScienceComputer Science (R0)