Abstract
Multi-class tissue classification from histological images is a complex challenge. The gold standard still relies on manual assessment by a trained pathologist, but it is a time-expensive task with issues about intra- and inter-operator variability. The rise of computational models in Digital Pathology has the potential to revolutionize the field. Historically, image classifiers relied on handcrafted feature extraction, combined with statistical classifiers, as Support Vector Machines (SVMs) or Artificial Neural Networks (ANNs). In recent years, there has been a tremendous growth in Deep Learning (DL), for all the image recognition tasks, including, of course, those concerning medical images. Thanks to DL, it is now possible to also learn the process of capturing the most relevant features from the image, easing the design of specialized classification algorithms and improving the performance. An important problem of DL is that it requires tons of training data, which is not easy to obtain in medical domain, since images have to be annotated by expert physicians. In this work, we extensively compared three classes of approaches for the multi-class tissue classification task: (1) extraction of handcrafted features with the adoption of a statistical classifier; (2) extraction of deep features using the transfer learning paradigm, then exploiting SVM or ANN classifiers; (3) fine-tuning of deep classifiers. After a cross-validation on a publicly available dataset, we validated our results on two independent test sets, obtaining an accuracy of 97% and of 77%, respectively. The second test set has been provided by the Pathology Department of IRCCS Istituto Tumori Giovanni Paolo II and has been made publicly available (http://doi.org/10.5281/zenodo.4785131).
N. Altini, T.M. Marvulli, M. Caputo, S. De Summa, F.A. Zito—Equally contributed to this paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Siegel, R.L., et al.: Colorectal cancer statistics, 2020. CA. Cancer J. Clin. 70, 145–164 (2020). https://doi.org/10.3322/caac.21601
Linder, N., et al.: Identification of tumor epithelium and stroma in tissue microarrays using texture analysis. Diagn. Pathol. 7, 22 (2012). https://doi.org/10.1186/1746-1596-7-22
Kather, J.N., et al.: Multi-class texture analysis in colorectal cancer histology. Sci. Rep. 6, 1–11 (2016). https://doi.org/10.1038/srep27988
Kather, J.N., et al.: Predicting survival from colorectal cancer histology slides using deep learning: a retrospective multicenter study. PLoS Med. 16, 1–22 (2019). https://doi.org/10.1371/journal.pmed.1002730
Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition, 1–14 (2014)
Howard, A.G., et al.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv Prepr. arXiv:1704.04861 (2017)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269. IEEE (2017). https://doi.org/10.1109/CVPR.2017.243
Kassani, S.H., Kassani, P.H., Wesolowski, M.J., Schneider, K.A., Deters, R.: Classification of histopathological biopsy images using ensemble of deep learning networks. In: CASCON 2019 Proc. - Conf. Cent. Adv. Stud. Collab. Res. - Proc. 29th Annu. Int. Conf. Comput. Sci. Softw. Eng., pp. 92–99 (2020)
Hochreiter, S., Schmidhuber, J.: Long Short-Term Memory. Neural Comput. 9, 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
Bychkov, D., et al.: Deep learning based tissue analysis predicts outcome in colorectal cancer. Sci. Rep. 8, 1–11 (2018). https://doi.org/10.1038/s41598-018-21758-3
Kather, J.N., et al.: Collection of textures in colorectal cancer histology (2016). https://doi.org/10.5281/zenodo.53169
Kather, J.N., Halama, N., Marx, A.: 100,000 histological images of human colorectal cancer and healthy tissue (2018). https://doi.org/10.5281/zenodo.1214456
Macenko, M., et al.: A method for normalizing histology slides for quantitative analysis. In: 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1107–1110 (2009). https://doi.org/10.1109/ISBI.2009.5193250
Altini, N., et al.: Pathologist’s annotated image tiles for multi- class tissue classification in colorectal cancer (2021). https://doi.org/10.5281/zenodo.4785131
Ahonen, T., Matas, J., He, C., Pietikäinen, M.: Rotation invariant image description with local binary pattern histogram fourier features. In: Salberg, A.-B., Hardeberg, J.Y., Jenssen, R. (eds.) SCIA 2009. LNCS, vol. 5575, pp. 61–70. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02230-2_7
Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24, 971–987 (2002). https://doi.org/10.1109/TPAMI.2002.1017623
Pietikäinen, M., Hadid, A., Zhao, G., Ahonen, T.: Computer vision using local binary patterns. Presented at the (2011). https://doi.org/10.1007/978-0-85729-748-8_14
Haralick, R.M., Dinstein, I., Shanmugam, K.: Textural features for image classification. IEEE Trans. Syst. Man Cybern. SMC-3, 610–621 (1973). https://doi.org/10.1109/TSMC.1973.4309314
Tamura, H., Mori, S., Yamawaki, T.: Textural features corresponding to visual perception. IEEE Trans. Syst. Man Cybern. 8, 460–473 (1978). https://doi.org/10.1109/TSMC.1978.4309999
Bianconi, F., Álvarez-Larrán, A., Fernández, A.: Discrimination between tumour epithelium and stroma via perception-based features. Neurocomputing 154, 119–126 (2015). https://doi.org/10.1016/j.neucom.2014.12.012
Tellez, D., et al.: Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Med. Image Anal. 58, 101544 (2019). doi:https://doi.org/10.1016/j.media.2019.101544
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015). https://doi.org/10.1038/nature14539
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE. 86, 2278–2324 (1998). https://doi.org/10.1109/5.726791
Krizhevsky, A., Sutskever, I., Hinton, G.E.: 2012 AlexNet. Adv. Neural Inf. Process. Syst. (2012). https://doi.org/10.1016/j.protcy.2014.09.007
Zeng, G., He, Y., Yu, Z., Yang, X., Yang, R., Zhang, L.: InceptionNet/GoogLeNet - going deeper with convolutions. CVPR 91, 2322–2330 (2016). https://doi.org/10.1002/jctb.4820
He, K., Girshick, R., Dollár, P.: Rethinking ImageNet Pre-training, pp. 1–10 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37, 1904–1916 (2015). https://doi.org/10.1109/TPAMI.2015.2389824
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2016-Decem, pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90.
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53
Araujo, T., et al.: Classification of breast cancer histology images using convolutional neural networks. PLoS ONE 12, 1–14 (2017). https://doi.org/10.1371/journal.pone.0177544
Altini, N., et al.: Semantic segmentation framework for glomeruli detection and classification in kidney histological sections. Electronics. 9, 503 (2020). https://doi.org/10.3390/electronics9030503
Altini, N., et al.: A deep learning instance segmentation approach for global glomerulosclerosis assessment in donor kidney biopsies. Electronics 9, 1768 (2020). https://doi.org/10.3390/electronics9111768
Weiss, K., Khoshgoftaar, T.M., Wang, D.: A survey of transfer learning. J. Big Data 3(1), 1–40 (2016). https://doi.org/10.1186/s40537-016-0043-6
Campanella, G., et al.: Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25, 1301–1309 (2019). https://doi.org/10.1038/s41591-019-0508-1.
Schmauch, B., et al.: A deep learning model to predict RNA-Seq expression of tumours from whole slide images. Nat. Commun. 11, 3877 (2020). https://doi.org/10.1038/s41467-020-17678-4
Fu, J., Singhrao, K., Cao, M., Yu, V., Santhanam, A.P., Yang, Y., Guo, M., Raldow, A.C., Ruan, D., Lewis, J.H.: Generation of abdominal synthetic CTs from 0.35T MR images using generative adversarial networks for MR-only liver radiotherapy. Biomed. Phys. Eng. Express. 6 (2020). https://doi.org/10.1088/2057-1976/ab6e1f.
Levy-jurgenson, A.: Spatial Transcriptomics Inferred from Pathology Whole-Slide Images Links Tumor Heterogeneity to Survival in Breast and Lung Cancer. 1–16 (2020).
Cascianelli, S., et al.: Dimensionality reduction strategies for CNN-based classification of histopathological images. In: De Pietro, G., Gallo, L., Howlett, R.J., Jain, L.C. (eds.) KES-IIMSS 2017. SIST, vol. 76, pp. 21–30. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-59480-4_3
Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., Liu, C.: A survey on deep transfer learning. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) ICANN 2018. LNCS, vol. 11141, pp. 270–279. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01424-7_27
Komura, D., Ishikawa, S.: Machine learning methods for histopathological image analysis. Comput. Struct. Biotechnol. J. 16, 34–42 (2018). https://doi.org/10.1016/j.csbj.2018.01.001
der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, (2008)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vision 128(2), 336–359 (2019). https://doi.org/10.1007/s11263-019-01228-7
Deng, J., Li, K., Do, M., Su, H., Fei-Fei, L.: Construction and analysis of a large scale image ontology. Presented at the (2009)
Acknowledgments
This research was funded by Italian Apulian Region “Tecnopolo per la medicina di precisione”, CUP B84I18000540002.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Altini, N. et al. (2021). Multi-class Tissue Classification in Colorectal Cancer with Handcrafted and Deep Features. In: Huang, DS., Jo, KH., Li, J., Gribova, V., Bevilacqua, V. (eds) Intelligent Computing Theories and Application. ICIC 2021. Lecture Notes in Computer Science(), vol 12836. Springer, Cham. https://doi.org/10.1007/978-3-030-84522-3_42
Download citation
DOI: https://doi.org/10.1007/978-3-030-84522-3_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-84521-6
Online ISBN: 978-3-030-84522-3
eBook Packages: Computer ScienceComputer Science (R0)