research-article

Generative Adversarial Training for Weakly Supervised Nuclei Instance Segmentation

Authors:

Huanhuan Sheng,

Yuan WenAuthors Info & Claims

2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

Pages 3649 - 3654

https://doi.org/10.1109/SMC42975.2020.9283412

Published: 11 October 2020 Publication History

Abstract

Nuclei segmentation occupies an important position in medical image analysis, which helps to predict and diagnose diseases. With the further research of deep learning, the task of nuclei segmentation has been automated. However, most existing methods require a great deal of manually marked full masks for training, which is time-consuming and labor-intensive, and can only be done by professional personnel. For the purpose of reducing the cost of labeling, we propose a weakly supervised method using generative adversarial training for segmentation of nucleus. In the case of no boundary, but only the centroid of the nucleus, the proposed method segmented the nucleus region with blurred boundaries. We first use the generative adversarial network(GAN) to generate the likelihood map of the nuclear centroid, then use Guided Backpropagation to visualize the pixels that contributes to the detection of the centroid of each nucleus, and finally obtain the segmentation mask of the nucleus by graph-cut. In addition, for the purpose of training the network better, we performed stain normalization on each pathological image. We have verified the proposed method on a multi-organ nuclei dataset. The final experiment results show that our advanced method achieves better segmentation performance than other weakly supervised methods, and can even reach the level of full supervision.

References

[1]

G. Eason, B. Noble, and I. N. Sneddon, “On certain integrals of Lipschitz-Hankel type involving products of Bessel functions,” Phil. Trans. Roy. Soc. London, vol. A247, pp. 529–551, April 1955. (references)

[2]

Rojas-Moraleda R, Xiong W, and Halama N, “Robust detection and segmentation of cell nuclei in biomedical images based on a computational topology framework,” Medical image analysis, vol. 38, pp. 90–103, 2017.

[3]

Peter Naylor, Marick Laé, Fabien Reyal, and Thomas Walter, “Segmentation of nuclei in histopathology images by deep regression of the distance map,” IEEE Transactions on Medical Imaging, 2018.

[4]

Zhang D, Song Y, and Liu S, “Nuclei instance segmentation with dual contour-enhanced adversarial network,” 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE, 2018, pp. 409–412.

[5]

Mahmood F, Borders D, and Chen R, “Deep adversarial training for multi-organ nuclei segmentation in histopathology images,” IEEE transactions on medical imaging, 2019.

[6]

Chamanzar, Alireza, and Yao Nie. “Weakly Supervised Multi-Task Learning for Cell Detection and Segmentation,” 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE, 2020.

[7]

Rajchl M, Lee M C H, and Oktay O, “Deepcut: Object segmentation from bounding box annotations using convolutional neural networks,” IEEE transactions on medical imaging, vol. 36, no. 2, pp. 674–683, 2016.

[8]

Martin Rajchl, Matthew CH Lee, Ozan Oktay, Konstantinos Kamnitsas, Jonathan Passerat- Palmbach, Wenjia Bai, Mellisa Damodaram, Mary A Rutherford, Joseph V Hajnal, and Bernhard Kainz, “Deepcut: Object segmentation from bounding box annotations using convolutional neural networks,” IEEE transactions on medical imaging, vol. 36, no. 2, pp. 674–683, 2017.

[9]

Ahn J, and Kwak S, “Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4981–4990, 2018.

[10]

Lin D, Dai J, and Jia J, “Scribblesup: Scribble-supervised convolutional networks for semantic segmentation,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3159–3167, 2016.

[11]

S. Dutt Jain, and K. Grauman, “Active image segmentation propagation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 2864–2873, 2016.

[12]

L. Yang, Y. Zhang, J. Chen, S. Zhang, and D. Z. Chen, “Suggestive annotation: A deep active learning framework for biomedical image segmentation,” in Proc. Int. Conf. Med. Image Comput. Comput. Assist. Intervention, pp. 399–407, 2017.

[13]

Yousef Al-Kofahi, Wiem Lassoued, William Lee, and Badrinath Roysam, “Improved automatic detection and segmentation of cell nuclei in histopathology images,” IEEE Transactions on Biomedical Engineering, vol. 57, no. 4, pp. 841–852, 2010.

[14]

Mitko Veta, Paul J Van Diest, Robert Kornegoor, André Huisman, Max A Viergever, and Josien PW Pluim, “Automatic nuclei segmentation in H&E stained breast cancer histopathology images,” PloS one, vol. 8, no. 7, pp. e70221, 2013.

[15]

Fuyong Xing, Yuanpu Xie, and Lin Yang, “An automatic learning-based framework for robust nucleus segmentation,” IEEE transactions on Medical Imaging, vol.35, no. 2, pp. 550–566, 2016.

[16]

Hui Qu, Gregory Riedlinger, Pengxiang Wu, Qiaoying Huang, Jingru Yi, Subhajyoti De, and Dimitris Metaxas, “Joint segmentation and fine-grained classification of nuclei in histopathology images,” 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy, pp. 900–904

[17]

Q Liang, Y Nan, G Coppola, K Zou, W Sun, D Zhang, Y Wang, and G Yu, “Weakly supervised biomedical image segmentation by reiterative learning,” IEEE Journal of biomedical and health informatics, vol. 23, no. 3, pp. 1205–1214, 2018.

[18]

Kervadec H, Dolz J, Tang M, Granger E, Boykov Y, and Ayed I, “Constrained-CNN losses for weakly supervised segmentation,” Medical image analysis, vol. 54, pp. 88–99, 2019.

[19]

Qu H, Wu P, Huang Q, et al. Weakly supervised deep nuclei segmentation using points annotation in histopathology images[C]//International Conference on Medical Imaging with Deep Learning. 2019: 390–400.

[20]

Nishimura K, and Bise R, “Weakly Supervised Cell Instance Segmentation by Propagating from Detection Response,” International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 649–657, 2019.

[21]

Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, David W, Sherjil O, Aaron C, Yoshua B, “Generative adversarial nets,” Advances in neural information processing systems, pp. 2672–2680, 2014.

Digital Library

[22]

Schlegl T, Seeböck P, and Waldstein S M, “Unsupervised anomaly detection with generative adversarial networks to guide marker discovery,” International conference on information processing in medical imaging, pp. 146–157, 2017.

[23]

V. Alex, M. S. KP, S. S. Chennamsetty, and G. Krishnamurthi, “Generative adversarial networks for brain lesion detection,” Medical Imaging 2017: Image Processing. International Society for Optics and Photonics, pp. 101330G, 2017.

[24]

M. Rezaei, H. Yang, and C. Meinel, “Whole heart and great vessel segmentation with context-aware of generative adversarial networks,” in Bildverarbeitung für die Medizin 2018, pp. 353–358, 2018.

[25]

Kim T, Cha M, and Kim H, “Learning to discover cross-domain relations with generative adversarial networks,” Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 1857–1865, 2017.

[26]

F. Mahmood, R. Chen, and N. J. Durr, “Unsupervised reverse domain adaptation for synthetic medical images via adversarial training,” IEEE Transactions on Medical Imaging, 2018.

[27]

A. Madani, M. Moradi, A. Karargyris, and T. Syeda-Mahmood, “Semisupervised learning with generative adversarial networks for chest x-ray classification with ability of data domain adaptation,” in Biomedical Imaging (ISBI 2018), pp. 1038–1042, 2018.

[28]

Zhu J Y, Park T, Isola P, Efros A, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” Proceedings of the IEEE international conference on computer vision. pp. 2223–2232, 2017.

[29]

Isola P, Zhu J Y, Zhou T, Efros A, “Image-to-image translation with conditional adversarial networks,” Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125-1134, 2017.

[30]

Kumar N, Verma R, and Anand D, “A multi-organ nucleus segmentation challenge,” IEEE transactions on medical imaging, 2019.

[31]

N. Kumar, R. Verma, S. Sharma, S. Bhargava, A. Vahadane, and A. Sethi, “A dataset and a technique for generalized nuclear segmentation for computational pathology,” IEEE Transactions on Medical Imaging, vol. 36, no. 7, pp. 1550–1560, July 2017.

[32]

Zhou Y, Zhu Y, and Ye Q, “Weakly supervised instance segmentation using class peak response,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3791–3800, 2018.

[33]

Ronneberger O, Fischer P, and Brox T, “U-net: Convolutional networks for biomedical image segmentation” International Conference on Medical image computing and computer-assisted intervention, pp. 234–241, 2015.

[34]

A. Vahadane, T. Peng, A. Sethi, S. Albarqouni, L. Wang, M. Baust, K. Steiger, A. M. Schlitter, I. Esposito, and N. Navab, “Structurepreserving color normalization and sparse stain separation for histological images,” IEEE transactions on medical imaging, vol. 35, no. 8, pp. 1962–1971, 2016.

[35]

Zilong Hu, Jinshan Tang, Ziming Wang, Kai Zhang, Lin Zhang, Qingling Sun, “Deep Learning for Image-based Cancer Detection and Diagnosis — A Survey,” Pattern Recognition, vol. 83, pp. 134–149, November 2018.

Digital Library

Cited By

Meng XZou T(2023)Clinical applications of graph neural networks in computational histopathologyComputers in Biology and Medicine10.1016/j.compbiomed.2023.107201164:COnline publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1016/j.compbiomed.2023.107201
Alamir MAlghamdi M(2022)The Role of Generative Adversarial Network in Medical Image Analysis: An In-depth SurveyACM Computing Surveys10.1145/352784955:5(1-36)Online publication date: 3-Dec-2022
https://dl.acm.org/doi/10.1145/3527849

Recommendations

Learning with Noise: Mask-Guided Attention Model for Weakly Supervised Nuclei Segmentation
Medical Image Computing and Computer Assisted Intervention – MICCAI 2021
Abstract
Deep convolutional neural networks have been highly effective in segmentation tasks. However, high performance often requires large datasets with high-quality annotations, especially for segmentation, which requires precise pixel-wise labelling. ...
Self-supervised Pre-training for Nuclei Segmentation
Medical Image Computing and Computer Assisted Intervention – MICCAI 2022
Abstract
The accurate segmentation of nuclei is crucial for cancer diagnosis and further clinical treatments. For semantic segmentation of nuclei, Vision Transformers (VT) have the potentiality to outperform Convolutional Neural Network (CNN) based models ...
Semi- and Weakly- Supervised Semantic Segmentation with Deep Convolutional Neural Networks
MM '15: Proceedings of the 23rd ACM international conference on Multimedia

Successful semantic segmentation methods typically rely on the training datasets containing a large number of pixel-wise labeled images. To alleviate the dependence on such a fully annotated training dataset, in this paper, we propose a semi- and weakly-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

Oct 2020

4507 pages

Copyright © 2020.

Publisher

IEEE Press

Publication History

Published: 11 October 2020

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Meng XZou T(2023)Clinical applications of graph neural networks in computational histopathologyComputers in Biology and Medicine10.1016/j.compbiomed.2023.107201164:COnline publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1016/j.compbiomed.2023.107201
Alamir MAlghamdi M(2022)The Role of Generative Adversarial Network in Medical Image Analysis: An In-depth SurveyACM Computing Surveys10.1145/352784955:5(1-36)Online publication date: 3-Dec-2022
https://dl.acm.org/doi/10.1145/3527849

View Options

View options

Media

Figures

Other

Tables

View Table of Contents