Nothing Special   ยป   [go: up one dir, main page]

Skip to main content

Semi-supervised Training of Optical Flow Convolutional Neural Networks in Ultrasound Elastography

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention โ€“ MICCAI 2020 (MICCAI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12263))

Abstract

Convolutional Neural Networks (CNN) have been found to have great potential in optical flow problems thanks to an abundance of data available for training a deep network. The displacement estimation step in UltraSound Elastography (USE) can be viewed as an optical flow problem. Despite the high performance of CNNs in optical flow, they have been rarely used for USE due to unique challenges that both input and output of USE networks impose. Ultrasound data has much higher high-frequency content compared to natural images. The outputs are also drastically different, where displacement values in USE are often smooth without sharp motions or discontinuities. The general trend is currently to use pre-trained networks and fine-tune them on a small simulation ultrasound database. However, realistic ultrasound simulation is computationally expensive. Also, the simulation techniques do not model complex motions, nonlinear and frequency-dependent acoustics, and many sources of artifact in ultrasound imaging. Herein, we propose an unsupervised fine-tuning technique which enables us to employ a large unlabeled dataset for fine-tuning of a CNN optical flow network. We show that the proposed unsupervised fine-tuning method substantially improves the performance of the network and reduces the artifacts generated by networks trained on computer vision databases.

Supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) RGPIN-2020-04612.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Azizi, S., et al.: Learning from noisy label statistics: detecting high grade prostate cancer in ultrasound guided biopsy. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-Lรณpez, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 21โ€“29. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_3

    Chapter  Google Scholar 

  2. Zhou, S.K., Rueckert, D., Fichtinger, G.: Handbook of medical imagecomputing and computer assisted intervention. Academic Press, (2019)

    Google Scholar 

  3. Zhuang, B., Rohling, R., Abolmaesumi, P.: Region-of-interest-based closed-loop beamforming for spinal ultrasound imaging. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 66(8), 1266โ€“1280 (2019)

    Article  Google Scholar 

  4. Ophir, J., et al.: Elastography: ultrasonic estimation and imaging of the elastic properties of tissues. Proc. Inst. Mech. Eng. Part H: J. Eng. Med. 213(3), 203โ€“233 (1999)

    Article  Google Scholar 

  5. Dosovitskiy, A., et al.: Flownet: learning optical flow with convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp. 2758โ€“2766 (2015)

    Google Scholar 

  6. Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., Brox, T.: Flownet 2.0: evolution of optical flow estimation with deep networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2462โ€“2470 (2017)

    Google Scholar 

  7. Sun, D., Yang, X., Liu, M.-Y., Kautz, J.: Pwc-net: cnns for optical flow using pyramid, warping, and cost volume. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8934โ€“8943 (2018)

    Google Scholar 

  8. Hui, T.W., Tang, X., Change Loy, C.: Liteflownet: a lightweight convolutional neural network for optical flow estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8981โ€“8989 (2018)

    Google Scholar 

  9. Kibria, M.G., Rivaz, H.: GLUENet: ultrasound elastography using convolutional neural network. In: Stoyanov, D., et al. (eds.) POCUS/BIVPCS/CuRIOUS/CPM -2018. LNCS, vol. 11042, pp. 21โ€“28. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01045-4_3

    Chapter  Google Scholar 

  10. Peng, B., Xian, Y., Jiang, J.: A convolution neural network-based speckletracking method for ultrasound elastography. In: 2018 IEEEInternational Ultrasonics Symposium (IUS), pp. 206โ€“212. IEEE (2018)

    Google Scholar 

  11. Tehrani, A.K., Rivaz, H.: Displacement estimation in ultrasound elastography using pyramidal convolutional neural network. In: IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, (2020)

    Google Scholar 

  12. Wu, S., Gao, Z., Liu, Z., Luo, J., Zhang, H., Li, S.: Direct reconstruction of ultrasound elastography using an end-to-end deep neural network. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-Lรณpez, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 374โ€“382. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_43

    Chapter  Google Scholar 

  13. Peng, B., Xian, Y., Zhang, Q., Jiang, J.: Neural network-based motion tracking for breast ultrasound strain elastography: an initial assessment of performance and feasibility. Ultrason. Imaging. 42(2), 74โ€“91 (2020)

    Article  Google Scholar 

  14. Gao, Z., et al.: Learning the implicit strain reconstruction in ultrasound elastography using privileged information. Med. Image Anal. 58, 11โ€“18 (2019)

    Article  Google Scholar 

  15. Evain, E., Faraz, K., Grenier, T., Garcia, D., De Craene, M., Bernard, O.: A pilot study on convolutional neural networks for motion estimation from ultrasound images. Ferroelectrics, and Frequency Control, IEEE Transactions on Ultrasonics, (2020)

    Google Scholar 

  16. Li, Z., Hoiem, D.: Learning without forgetting. IEEE Trans. Pattern Anal. Mach. Intell. 40(12), 2935โ€“2947 (2017)

    Article  Google Scholar 

  17. Sandrin, L., et al.: Transient elastography: a new noninvasive method for assessment of hepatic fibrosis. Ultrasound Med. Biol. 29(12), 1705โ€“1713 (2003)

    Article  Google Scholar 

  18. Godard, C., Mac Aodha, O., Brostow, G.J.: Unsupervised monocular depth estimation with left-right consistency. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 270โ€“279 (2017)

    Google Scholar 

  19. Meister, S., Hur, J., Roth, S.: Unflow: unsupervised learning of optical flow with a bidirectional census loss. In: Thirty-Second AAAI Conference on Artificial Intelligence, (2018)

    Google Scholar 

  20. Ren, Z., Yan, J., Yang, X., Yuille, A., Zha, H.: Unsupervised learning of optical flow with patch consistency and occlusion estimation. Pattern Recogn. 103, 107191 (2020)

    Article  Google Scholar 

  21. Wang, Y., Yang, Y., Yang, Z., Zhao, L., Wang, P., Xu, W.: Occlusion aware unsupervised learning of optical flow. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4884โ€“4893 (2018)

    Google Scholar 

  22. Sun, D., Roth, S., Black, M.J.: A quantitative analysis of current practices in optical flow estimation and the principles behind them. Int. J. Comput. Vision 106(2), 115โ€“137 (2014)

    Article  Google Scholar 

  23. Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 801, pp. 151โ€“158. Springer, Heidelberg (1994). https://doi.org/10.1007/BFb0028345

    Chapter  Google Scholar 

  24. Hashemi, H.S., Rivaz, H.: Global time-delay estimation in ultrasound elastography. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 64(10), 1625โ€“1636 (2017)

    Article  Google Scholar 

  25. Mirzaei, M., Asif, A., Rivaz, H.: Combining total variation regularization with window-based time delay estimation in ultrasound elastography. IEEE Trans. Med. Imaging 38(12), 2744โ€“2754 (2019)

    Article  Google Scholar 

  26. Chen, T., Xu, B., Zhang, C., Guestrin, C.: Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174 (2016)

  27. Rivaz, H., Boctor, E.M., Choti, M.A., Hager, G.D.: Real-time regularized ultrasound elastography. IEEE Trans. Med. Imaging 30(4), 928โ€“945 (2011)

    Article  Google Scholar 

Download references

Acknowledgement

We thank NVIDIA for the donation of the GPU. The in vivo data was collected at Johns Hopkins Hospital. We thank E. Boctor, M. Choti and G. Hager for giving us access to this data.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ali K. Z. Tehrani .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

ยฉ 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

K. Z. Tehrani, A., Mirzaei, M., Rivaz, H. (2020). Semi-supervised Training of Optical Flow Convolutional Neural Networks in Ultrasound Elastography. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention โ€“ MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12263. Springer, Cham. https://doi.org/10.1007/978-3-030-59716-0_48

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-59716-0_48

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-59715-3

  • Online ISBN: 978-3-030-59716-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics