Breast histopathological image analysis using image processing techniques for diagnostic purposes: A methodological review

Download PDF

12k Accesses
45 Citations
2 Altmetric
Explore all metrics

Abstract

Breast cancer in women is the second most common cancer worldwide. Early detection of breast cancer can reduce the risk of human life. Non-invasive techniques such as mammograms and ultrasound imaging are popularly used to detect the tumour. However, histopathological analysis is necessary to determine the malignancy of the tumour as it analyses the image at the cellular level. Manual analysis of these slides is time consuming, tedious, subjective and are susceptible to human errors. Also, at times the interpretation of these images are inconsistent between laboratories. Hence, a Computer-Aided Diagnostic system that can act as a decision support system is need of the hour. Moreover, recent developments in computational power and memory capacity led to the application of computer tools and medical image processing techniques to process and analyze breast cancer histopathological images. This review paper summarizes various traditional and deep learning based methods developed to analyze breast cancer histopathological images. Initially, the characteristics of breast cancer histopathological images are discussed. A detailed discussion on the various potential regions of interest is presented which is crucial for the development of Computer-Aided Diagnostic systems. We summarize the recent trends and choices made during the selection of medical image processing techniques. Finally, a detailed discussion on the various challenges involved in the analysis of BCHI is presented along with the future scope.

Breast cancer image analysis using deep learning techniques – a survey

Article 21 October 2022

Survey of deep learning in breast cancer image analysis

Article 24 August 2019

A review on machine learning techniques for the assessment of image grading in breast mammogram

Article 01 April 2022

Use our pre-submission checklist

Avoid common mistakes on your manuscript.

Introduction

Cancer is a significant medical issue worldwide and is a major public health concern [1]. Among all the types of cancers, breast cancer in women around the world is the second most common cancer [2,3,4]. Breast cancer is a malignant lesion formation in the breast region. Early detection of breast cancer helps in better selection of treatment and prevents risk on human life. However, biopsy followed by histopathological analysis is the only way to determine with assurance the tumor is benign or malignant since it reveals the microscopic structure of the tissues. A histopathological analysis is a procedural work carried out in pathology laboratories to study manifestation of diseases in the tissues. Fine needle aspiration cytology is an alternative approach to histopathological image analysis that studies the structure and characteristics of cells. Fine needle aspiration cytology reveals the presence of tumour. However, the type of tumour cannot be decided by this analysis as it is not highly sensitive. Histopathological imaging has been considered as the gold standard in recognizing almost all sorts of cancers since it captures a more detailed view of the diseases [5, 6]. For accurate identification of breast cancer, biopsy accompanied by microscopic examination is an essential aspect. In a biopsy, a small section of tissue from the suspicious region of the body is removed, processed, and dyed with Hematoxylin and Eosin (H&E) stains. Hematoxylin stains the nuclei to dark purple or blue and Eosin stains other structures into shades of pink, red, and orange [7]. Subsequently, a pathologist studies the structure of the tissue from the stained glass slides to differentiate them as benign or malignant. Histopathological images of the breast are analyzed at different magnifications to study the cellular and tissue level variations [8]. For example, at 100x magnification, the tissue patterns and distributions are studied while at 400x magnification cytological features such as shape and size of the nuclei, hyperchromatic nuclei, mitotic cell, and prominent nuclei [9] are studied. Based on these features, pathologists classify the tumor slides as benign and malignant. In case of malignancy further examination is performed to grade the tumor and suitable treatment is given to the affected individual. Breast cancer can be of different types and each of these types has different microscopic features. Figure 1 shows the histopathological types of breast cancer.

During the manual evaluation, a pathologist examines the morphological features like shape, size, and color of the Region Of Interest (ROI) such as nuclei. Any change from the expected normal appearance of the nuclei is considered abnormal and further evaluation is carried out to confirm it as a malignant condition. In some cases, the pathologist also needs to report the tumor grading to know the aggressiveness of cancer [11, 12]. Figure 2 shows the microscopic patterns of benign breast tumors. Figures 3 and 4 represents the microscopic patterns of malignant breast tumor.

From the past few decades, pathology laboratories are moving from optical microscopy to fully digital microscopy [13,14,15]. The evaluation is subjective and it may vary among pathologists and also among laboratories often leading to variability. The evaluation is also dependent on the experience and skill of the pathologist, instruments, staining procedure, and the approaches used to analyze the histopathological images. Manual evaluation of histopathological slides is a tedious and highly time-consuming task [7, 12]. Hence, there is a need for the design of an automation system to match the human evaluation process to diagnose abnormal cases correctly and act as a decision support system. Various image processing techniques are used to analyze images for evaluating the disease and prognosis. The basic steps involved in the development of Computer Aided Diagnostic (CAD) systems for histopathological images are pre-processing (color normalization), image segmentation, feature extraction, and classification [7, 8]. However, analyzing histopathological images is a challenging task in medical image processing due to the complex appearance, inconsistent staining, variation in illumination, overlapping and clustered nuclei and poorly fixed tissue samples [16, 17]. In the tissue preparation procedure, staining can also be affected by various determinants including the tissue itself, the thickness of the tissue section, the length of time at which tissue is exposed to stains, tissue foldings, artifacts in the stains [13], air bubbles [18] and blurring as shown in Fig. 5. All these factors result in poor segmentation and classification in the development of CAD systems. Over the past decades, much research is focused on the analysis of BCHI to bring automation to classify the image as benign and malignant using image processing and Machine Learning (ML) techniques [19,20,21]. This paper aims to provide a review of different attempts for automation of diagnosis based on histopathological image analysis using image processing techniques. In literature, several works [11, 12, 16, 21,22,23] reviewed breast histopathological image analysis. However, to the best of our knowledge, they focus only on a particular aspect of image analysis techniques such as color normalization [22, 23], segmentation [16] or classification [21]. In contrast to these works, the present paper focuses on all aspects of BCHI analysis such as datasets, color normalization, detection and segmentation of the potential ROI, feature extraction, and classification of histopathological breast images. This paper also reports the research gaps and concludes with an opinion on future work. The paper is restricted only to the review of the BCHI. The contributions of this review paper are as follows:

The state-of-the review articles address a specific aspect of the problem such as segmentation of potential regions of interest or classification. However, we have summarized all the steps of BCHI analysis such as pre-processing, segmentation, feature extraction, and classification. Further, we have also summarized traditional and Deep Learning (DL) based methods to process BCHI.
Developing DL models dependent on the availability of large datasets with annotations. In this regard, we have summarized various publicly available datasets.
A summary on recent trends and popular choices of methods for various steps in processing BCHIs is provided.
Finally, we have summarized, our observations, all the challenges of processing images along with the future direction.

This paper is structured as follows; “Publicly available datasets” gives details about publicly available datasets for BCHI analysis. “Overview of the review articles on automation of histopathological image analysis” summarizes the review that was carried out in the field of histopathological image analysis. The review of various image analysis approaches used for automation is discussed in “Image processing approaches”. “Discussion” and “Challenges” briefs about the discussion and the future work. Finally, the paper ends with a conclusion.

Publicly available datasets

To develop a robust CAD system for breast cancer detection using histopathological images, it is necessary to have image datasets. The details of the publicly available datasets for BCHI analysis are given in Table 1. Details such as the total number of images, magnification factors, image size, image format, and classes are highlighted. These image datasets can be used by the researchers to develop an algorithm for the classification of Breast Cancer Histopathological Images (BCHI). However, all these datasets provide annotations at the image level for image classification. This limits the application of image processing methods and analysis of histopathological images at the pixel-level.

BreakHis [24] database is the most popularly used database among the research community for classification. BreakHis database contains a total of 7909 images collected from 82 patients organized into four different magnification factors namely 40X, 100X, 200X and 400X. A total of 2480 images of benign and 5429 images of malignant cases of various subtypes of BCHI are provided. The histopathological types of benign breast tumors are adenosis, fibroadenoma, phyllodes tumor, and tubular adenoma. The types of malignant breast tumors are ductal carcinoma, lobular carcinoma, mucinous carcinoma, and papillary carcinoma. This dataset allows the researchers to address the problems in terms of binary and multi-class classification tasks and also at different magnifications.

The second popularly used database among the research community is BreAst Cancer Histology (BACH) dataset [25]. The dataset is composed of microscopy images and Whole Slide Images (WSIs). A total of 400 microscopy images and 30 WSIs of multiple regions namely normal, benign, in situ carcinoma, and invasive carcinoma are provided. MITOS-ATYPIA-14 [26] is the most frequently used database for the detection and classification of mitosis and non-mitosis [27]. BACH [25], TUPAC-2016 [27], Camelyon 16 [28] and Bioimaging 2015 [29] databases are also used by the researchers for the analysis of BCHI.

Table 1 Overview of the publicly available BCHI datasets

Full size table

Overview of the review articles on automation of histopathological image analysis

From the literature, it is observed that a few researchers have carried out a review on histopathological image analysis [11, 12, 35], stain normalization [22, 23], segmentation [16] and classification [21]. The details about the review on the histopathological image analysis are discussed and summarized in this section.

Gurcan et al. [11] summarized the recent state-of-the-art CAD technology for histopathological image analysis. The authors have also emphasized the usage of standard datasets for the evaluation of developed CAD systems since it helps in easier analysis and comparison. Veta et al. [12] reviewed various methods proposed for the analysis of BCHI. The authors discussed the complexity of the tissue characteristics that need to be studied to improve the robustness of the system. The authors in [17, 19,20,21] reviewed the usage of ML techniques for the analysis of histopathological images. In [20] and [21], a summary of the available dataset for breast cancer analysis and generalized image classification techniques like supervised, unsupervised, and DL classifiers is provided. The different approaches used for the histopathological image analysis like nuclei detection, segmentation, feature extraction, and classification were reviewed by Irshad et al. [16]. They also discussed a few benchmark datasets, the problems and challenges of microscopic image segmentation, and mentioned the issue of robustness in terms of clinical and technical conditions. He et al. [7] discussed the characteristics of the histology images and reviewed the state-of-the-art methods used for image segmentation techniques for feature extraction and disease classification. In [8] the authors reviewed the computational steps required to automatically diagnose cancer in histopathological images. In their review, they investigated the types of features that are used in the diagnosis of different types of cancer. Janowczyk et al. [36] investigated how DL approaches can be used in the digital pathology domain. The study was conducted on the set of use cases for segmentation, detection, and classification. It has been suggested that the quality of the classifiers can be improved by utilizing hand-crafted features along with the DL approach. Various image analysis methodologies in histology image analysis were surveyed by Loukas et al. [37]. The authors described the cell detection problem and also listed out the limitations that need to be addressed. Fuchs et al. [38] reported the challenges involved in computational pathology workflow. They discussed the future directions in research for diagnostic ML. State-of-the-art methods and applications involved in large-scale medical image analytics were summarized by Zhang et al. [39]. Litjens et al. [40] surveyed the DL techniques in the domain of medical image analysis. The authors discussed the state-of-the-art DL approaches and challenges involved in the analysis of BCHI. A comprehensive survey on the automatic diagnosis of breast cancer using DL techniques on the BreakHis dataset was provided by Benhammou et al. [41]. They also explored the DL technique for magnification-independent multi-category classification problems. The authors in [42] presented a review on lymph node assistant for breast cancer images. The findings from a multi-reader and multi-case study of pathologists utilizing state-of-art algorithms are summarized. Debelle et al. [43] reviewed state-of-the-art DL algorithms used for the detection of breast cancer. Srinidhi et al. [44] presented a comprehensive overview of deep neural network architectures developed for analyzing histopathological images and also outlined some of the issues and future trends. The ML and DL approach for diagnosis of breast carcinoma were surveyed in [45] where they discussed the issues involved in the development of CAD systems. Further, they analyzed various ML and DL approaches for cancer diagnosis.

From the literature, it is seen that a considerable amount of review is carried out on the various aspects of automation of histopathological images. However, there is a lack of detailed review focusing on all aspects of histopathological image analysis such as color normalization, detection, and segmentation of the potential ROI, feature extraction, and classification in literature. To the best of our knowledge, these review papers are specific and focused on a particular aspect of breast histopathological image analysis.

Image processing approaches

Histopathological image analysis aims to classify the images as malignant and benign and act as a decision support system. Analysis of histopathological images by using image processing techniques involves various steps such as color normalization, segmentation, feature extraction, and classification. This pipeline of operation is popularly used in traditional image processing techniques while modern approaches based on DL use end-to-end style learning. In the traditional approach, color normalization step is used as a pre-processing step to remove the variations in color and illumination. Subsequently, a segmentation algorithm is applied to identify potential regions of interest. Feature descriptors are defined to extract the most discriminative features and finally, an ML algorithm is trained to classify the images into different classes based on the extracted features.

This section initially presents various color normalization techniques developed followed by detection and segmentation of the potential regions of interest. Subsequently, a summary of various feature extraction and classification techniques used for BCHI is presented.

Color normalization

Due to inconsistency in staining and image acquisition, histopathological images suffer from color and illumination variations. The main factor that influences the color variation is the difference in staining procedures adopted in different laboratories [22, 23, 46]. The different scanners and types of equipment used to capture the image results in illumination variation. Ignoring the variance in color and illumination of histopathological images often leads to incorrect results [47]. To overcome these problems pre-processing step, known as color normalization is employed for the histopathological image analysis. An illustration of color and illumination variance of BCHI are shown in Fig. 6. In this section, a summary of various works on the color normalization techniques for histopathological images is presented.

Many attempts have been made to eliminate the color and illumination variations in histopathological images over the past years [48,49,50,51] , [52,53,54]. Systematic study on assigning of an image color to another image was carried out in 2001 by Reinhard et al. [55]. The color characteristics of a template image are brought to an input image based on mean and standard deviation in LAB color space representation. This method is popularly known as the Rehinard method. To improve the color consistency in histopathological images Basavanhally et al. [48] proposed an Expectation-Maximization (EM) based segmentation–driven Standardization (EMS) algorithm. In this approach, the input and the normalized image were qualitatively compared during the nuclei segmentation step. To obtain an accurate stain density map in histopathology images Vahadane et al. [51] proposed a Structure-Preserving Color Normalization (SPCN) algorithm. They converted the source and the target RGB images to optical density space using the Beer-Lambert law. The structure of the original image is preserved by employing the SPCN algorithm. But this method fails to preserve the color information of the source image and suffers from local minima. Roy et al. [50] developed a Fuzzy based Modified Reinhard (FMR) method to handle the color variations in H&E stained images. The conventional Reinhard method reduces the contrast of the source image. Hence to overcome this problem, fuzzy logic with the Reinhard method was incorporated. The authors of [49] experimented on different color normalization algorithms and proposed stain normalization using saturation-weighted statistics. Their method is capable of identifying the cause for color variation. An alternative reference space for color normalization in histopathological images was reported by [54]. Euclidean distance was utilized to measure the intra-cluster and inter-cluster ratios. Each eight-bit input image is reduced to ten colors using K-means clustering. To overcome inconsistency in the staining process, Macenko et al. [52] proposed a stain quantization algorithm that belongs to a class of unsupervised normalization approaches. The images were analyzed on the shape and stain-based features. Further, an algorithm to obtain the optical stain vector with various stain combinations was provided. Tosta et al. [53] developed an estimation method to handle the faded regions in the histopathological images. The downside of the proposed method is that fails to preserve the tissue color corrections resulting in low performance. A Quantile normalization approach was proposed by Cao et al. [56] to develop an automatic breast cancer grading system. Gadermayr et al. [57] investigated the various stain normalization techniques concerning tissue classification. The study included five different normalization techniques, followed by extraction of five features on two differently stained renal images on two experimental setups. Khan et al. [58] proposed a non-linear mapping technique for the normalization of an input image to the color distribution of the reference image based on color deconvolution. Bukenya [59] developed a hybrid technique for stain normalization which consists of two stages namely, the stain separation stage and color transfer stage. The input images and the reference image were converted from RGB to Optical Density space and performed stain separation and color transfer steps to normalize the image. A stain deconvolution approach by employing a multi-resolution wavelet representation of the image to evaluate the stain mixing matrix was presented by Alsubaie et al. [60]. They converted the input image to optical density image and extracted R, G, and B channels, and decomposed each color channel to its sub-band by employing wavelet decomposition. In the last few years, much study is carried out to address the problem of stain normalization in terms of processing time and additional system memory utilization. Anghel et al. [61] proposed an unsupervised approach for stain normalization in WSI. They adopted the method proposed in [52] to perform a high-performance stain normalization system and also proposed a method to detect low-quality images. The main objective of the study was on optimizing and enhancing the robustness of the stain normalization algorithm. An automated color segmentation approach was developed by Kothari et al. [62]. The experiment was performed on four different types of H&E stained images. The images were normalized by adopting two types of normalization techniques namely quantile normalization on all the pixels and normalization on the color maps of the images which are obtained by extracting the unique color. A color segmentation accuracy of 85% was reported. An experiment on extracting the color and texture information to evaluate the need for stain normalization was conducted by Gupta et al. [63]. The authors suggested a method for the selection of reference images. A comparative study on different normalization techniques for epithelium and stromal classification was performed by Sethi et al. [64]. A multi-resolution segmentation approach was performed for super pixel-based classification. The patch-based classification was performed by using Convolutional Neural Network (CNN). Bejnordi et al. [65] developed a fully automated algorithm called whole-slide image color standardizer for the standardization of whole-slide histopathological images. Color and spatial information were employed to classify the image pixel into distinctive stain components. In [66], a pipeline that utilizes an unsupervised method based on stain vector estimation was proposed to handle the memory and runtime bottlenecks in high magnification images. They claimed the method is computationally less expensive in terms of memory. Magliaro et al. [67] developed an open-source tool called Histology for the separation of dye colors in histology images. The graphical user interface was developed by utilizing the K-means clustering algorithm to isolate the dye colors in histological images. The performance of the tool was compared with the ImageJ color deconvolution plugin in terms of speed of color separation. An algorithm to reduce the stain variations in H&E stained histopathological images was proposed by Bejnordi et al. [46]. The standardization algorithm is based on the clustering of the images into two tissue components. Bautista et al. [68] implemented a color correction method by utilizing the color information of nine color patches of a color calibration slide. The proposed method does not work when there are color variations caused by the staining procedure. A flexible and robust image analysis algorithm was developed for the separation and quantification of immunohistochemical staining by Ruifrok et al. [69]. The proposed algorithm was designed to deconvolve the color information captured with RGB cameras and to estimate the contribution of each of the applied stains. A color transfer approach based on YCbCr color space for the enhancement of peripheral blood smear images was developed by Prasad et al. [70]. An approach for the selection of the template image and the effect of the good template image on color normalization was described. Clarke et al. [71] developed a color calibration assessment slide for digital pathology. The positive aspect of the method is the variations in the tissue thickness does not alter the shape of the spectrum. The authors also claimed that the proposed method will add to the reproducibility of the automation of image analysis systems. Bautista et al. [72] proposed a method for the detection and visualization of tissue folds in pre-scanned WSI. The proposed method incorporates the color enhancement technique which distinguishes between the folds and non-fold regions in an image. Color saturation and luminance of the image pixels with weighted differences are used as a shifting factor to the input RGB color of the image. In recent years color normalization was also carried out by using DL techniques [73, 74]. Generative Adversarial Networks (GANs) model for color normalization of H&E stained images was developed by Zanjani et al. [74]. The model has an end-to-end framework that was trained to learn the chromatin space of H&E images. Janowczyk et al. [73] proposed a stain normalization technique using Sparse Autoencoders (StaNoSA) to handle the color variations in H&E stained images. The StaNoSA was developed by incorporating sparse encoders as a core method to segregate the input images into distinct tissue classes. Hamidinekoo and Zwiggelaar [75] proposed a DL-based network to detect mitosis in BCHI. Color normalization was performed by utilizing the RGB histogram specification method. In this method, color values of the target image are normalized to the source image on a pixel-by-pixel basis.

Segmentation

Segmentation methods divide an image into a smaller group of pixels. Application of segmentation algorithms to BCHI aids in identifying the ROI . In this section, we provide details of the state-of-the-art methods for ROI detection and segmentation.

ROI detection and segmentation using a traditional approach

Segmentation of the ROI from histopathological images has received much attention in the past decade. Many approaches have been proposed in the literature for ROI detection and segmentation. Traditional image processing methods utilize thresholding [76], watershed transform [77,78,79], active contour models [80] and ML for nuclei segmentation. Figure 7 shows various types of nuclei in the BCHI. The state-of-the-art methods for ROI segmentation and detection using traditional approaches are summarized in this section.

Veta et al. [79] presented a marker-controlled watershed-based technique to segment the cancerous nuclei in whole digital slide images of the breast. A positive predictive value of 0.90, a sensitivity of 0.83, and dice coefficients of 0.9 were reported. Fatakdawala et al. [80] considered HER2+ breast histopathological images to detect the lymphocytes. They proposed expectation-maximization-driven geodesic active contour with overlap resolution. The method was evaluated on a total of 100 images were reported a detection sensitivity of over 86% and a positive predictive value of 64% respectively. Paramanandam et al. [81] proposed an integrated framework by utilizing a gradient-driven voting mechanism using 2D tensor voting along with the Markov Random Field loop backpropagation technique to segment the single nuclei breast images. A total of 8 H&E stained images along with two WSI were used in their study.

Wang et al. [82] developed an automatic nuclei segmentation method based on multi-scale region-growing with a double-strategy splitting model. To enhance the contrast between the nuclei and background, the top-hat transform method was used. Adaptive mathematical morphology and curvature scale space methods were used to separate the overlapped nuclei. A segmentation accuracy of 91% was reported. Dundar et al. [77] developed a method to classify intraductal breast lesions from histopathological images. The input images were converted from RGB to LAB space and segmentation was performed by using a watershed-based segmentation algorithm to segment the single cells. A total of 149 ROIs were used for testing and were reported an overall accuracy of 87%. Four different clustering algorithms are compared and studied for the task of nuclei segmentation by Kowal et al. [76] from fine needle biopsy images of the breast. Adaptive thresholding is used to segment the foreground and background objects. A clustering algorithm was applied to identify the nuclei in the foreground objects. Further, 42 morphological, texture, and topological features were extracted from the segmented nuclei. Three different classifiers were used for classification. A classification accuracy ranging from 96-98% was reported. Kost et al. [78] proposed a method for automatic nuclei detection using probability maps and watershed-based segmentation algorithms. Hematoxylin channel is extracted from the input images to generate a probability map, which represents the nuclei for each pixel. A threshold of 0.5 was utilized to detect the nuclei. An extended watershed-based algorithm was used to reduce over-segmentation and reported F1-scores ranging between 0.83-0.93. Vink et al. [83] proposed a method for detection of the nucleus from breast tissue using AdaBoost and active contour. They focused on the detection of nuclei of the epithelial cells, lymphocytes, and fibroblasts. The detection accuracy of 95% was reported. An integrated method for gland and nucleus detection and segmentation was proposed by Naik et al. [84]. The segmentation was performed using image information at three scales. The architectural features were used to distinguish cancer and non-cancer breast histopathological images. A total of 18 benign and 36 malignant images were considered. An accuracy of 81% using automatic segmentation and 77% with manually segmented structures were reported. Petushi et al. [85] performed segmentation by using adaptive thresholding and morphological operations on the grayscale image. Graph-based features were utilized to distinguish the lymphocyte and cancer nuclei by Basavanhally et al. [86]. Lymphocyte was detected and segmented using a segmentation scheme consisting of a Bayesian classifier and template matching for a total of 41 images. An unsupervised learning approach to identify the ROIs in the malignant samples of WSIs was proposed by Kumar and Prateek [87]. K-means clustering, morphological features, and shape features were utilized to identify the ROI. An accuracy of 85% was reported.

Bejnordi et al. [88] proposed a multi-scale superpixel classification algorithm for the identification of epithelial areas in WSIs. The ROI was segmented from the detected epithelial regions using a graph-based clustering algorithm. Authors in [89] proposed a method to differentiate between neoplastic epithelium and stromal reaction in breast carcinomas. Density-based clustering was performed on the centroid of the tumor cell to segment the neoplastic epithelium. The proposed algorithm was evaluated on 100 H&E stained images. An F1 score of 0.88 and a mean Jaccard index of 0.84 was reported. Paul and Mukherjee [90] reported a method to segment mitotic cells by adopting relative-entropy maximized scale space with morphological operations. Paramanandam et al. [91] focussed on the detection and segmentation of tubule regions in the images using K-means clustering algorithm. Colour-based segmentation and grid analysis was utilized to identify the nuclei regions. Filipczuk et al. [92] reported an automatic diagnostic approach for the analysis of fine needle biopsy images. Circular hough transform and Support Vector Machine (SVM) was used for the detection of nuclei. The model was evaluated on 737 microscopic images and reported an accuracy of 98% respectively.

ROI detection and segmentation using DL approach

Recently, DL-based algorithms have gained much popularity in the medical imaging community due to their end-to-end style learning. DL methods are attracting widespread interest in nucleus detection from histopathological images as they can learn deep features [93]. But these DL algorithms are dependent on the availability of large datasets with annotations. Several authors have utilized DL-based methods to segment nuclei from breast histopathological images [94,95,96,97,98,99]. The state-of-the-art methods for ROI segmentation and detection using DL approaches are summarized in this section. Naylor et al. [96] used deep neural network architecture with mathematical morphology for segmentation of nuclei. The three well-known architectures namely Pang Net, Fully Convolutional Net, and Deconv Net were utilized for semantic segmentation. A total of 33 images with 2754 annotated cell nuclei were used for the study. Segmentation results ranging from 76-94% were reported. A deep CNN-based shape recognition was introduced by Xing et al. [98] to generate initial shapes, which learn hierarchical feature representation from raw images. The algorithm was tested on three different types of pathology datasets. The precision of 71%, recall of 88%, and F1-score of 78% were reported. A Stacked Sparse Autoencoder with an encoder and decoder network for efficient detection of nuclei for histopathological images was developed [99]. From the pixel intensities, the model learns high-level features to distinguish the features of nuclei. F-measure of 84% and average area under the Precision-Recall curve of 78% were reported. Metha et al. [95] implemented an encoder and decoder model to semantically segment the tissue labels in breast images. A total of 240 breast biopsy images were used. An overall segmentation accuracy of 93% was reported.

Jung et al. [94] proposed the Mask R-CNN segmentation framework to achieve nuclei segmentation. Color normalization was performed by utilizing U-Net based deep convolution Gaussian mixture model. In the post-processing step, multiple inferences were utilized to improve segmentation performance. Wang et al. [97] developed a bending loss regularization architecture for nuclei segmentation. The network is composed of an encoder and a decoder model. The pre-trained 50 layer Residual Neural Network (ResNet) was used as an encoder. The nuclei instance branch and distance map branch were utilized in a decoder model. The study was performed on 21000 annotated nuclei from 30 images.

A method to segment the touching and overlapping nuclei was studied in [100,101,102,103,104,105]. In [103] Deep Convolutional Neural Network (DCNN) and marker-controlled watershed techniques were combined to segment the overlapping nuclei. Deep Interval-Marker-Aware network architecture was designed for learning the foreground, marker, and interval of nuclei. The logical operations were utilized to get foreground results of the nuclei from the learned interval between the overlapping nuclei. The results of the learned marker and nuclei segmentation refined by interval are passed to the marker-controlled watershed for separating the overlapping nuclei. Kumar et al. [101] performed nucleus segmentation using CNN. The architecture was designed to produce a ternary map that was able to identify inside and outside nucleus along with the nuclear boundary containing those between the touching and overlapping nuclei. The authors in [100] designed a CNN to detect the Invasive Ductal Carcinoma (IDC) tissue regions in the WSIs. A total of 162 WSI slides were used to evaluate the method. They reported an F-measure and accuracy of 71% and 84% respectively. Xu et al. [104] presented a CNN for the detection of nuclei, a region-based active contour method for segmentation, and adaptive ellipse fitting to handle the clustered and overlapping nuclei. The DL architectures such as Residual-inception-channel attention U-net [105], Atrous spatial pyramid pooling U-net [102], and conditional GAN [106] were also explored for the nuclei segmentation.

Several studies have been carried out to detect and segment mitosis in BCHI [107,108,109,110]. Wahab et al. [110] proposed a transfer learning-based deep CNN for segmentation and detection of mitotic nuclei. Pre-trained CNN’s were used for segmentation. They reported the detection rate with an F1-measure of 0.73 and area under the precision-recall curve as 76% respectively. Das and Dutta [107] reported a method to detect mitosis in BCHIs using DCNN model. The Haar wavelet was utilized to decompose the input image patches of 81 * 81 pixels to patches of \(21\times 21\) pixels. A total of 70 images were considered for training which has 720 mitotic cells and 30 images were considered for testing containing 200 mitotic cells. Precision, recall, and F1-score of 84%, 83%, and 85% were reported on the testing dataset. The authors in [109] proposed a Mask RCNN to automatically detect and segment mitosis in breast cancer slides. A multi-stage DL framework was reported by Li et al. [108] to detect the mitotic cells. The framework consists of a deep segmentation network, deep detection network, and deep verification network.

A study was also performed for automatic segmentation of carcinoma distribution in WSI of breast tissues [111]. Initially, WSI’s were split into patches and segmented by using DCNN along with encoder and decoder model. The merging technique based on fully connected Conditional Random Field was applied to combine the segmented patches. Segmentation accuracy and frequency weighted intersection over union (FWIoU) of 95% and 92% respectively were reported. The performance of a DL algorithm to detect the lymph node metastases in H&E stained breast cancer tissue sections was evaluated in [112].

A summary of the different segmentation techniques used in the literature for ROI detection along with the achieved performance metrics is given in Table 2. Figure 8 represents the different segmentation methods used in the literature. We have categorized the segmentation methods into threshold-based, region-based, clustering-based, fusion-based, and DL-based techniques. In the “Others” category, we have included the methods such as level set information and color-based segmentation. It is observed from Fig. 8, that most of the research groups used DL-based techniques for detection and segmentation of the ROIs in BCHI.

Table 2 Summary of the approaches used for ROI segmentation in BCHI

Full size table

Classification

Feature is defined as an “interesting” part of an image and helps in describing different regions in an image. These features help in identifying the same objects in different images. Classification is a process of assigning labels to different groups based on the identified features. In this section, we provide details of the state-of-the-art methods used for feature extraction and classification.

Classification using traditional image processing approach

In the last few years, many studies were carried out to extract different features from BCHI. The handcrafted features such as statistical features, different texture features, morphological features, and color features were studied to classify the images as benign and malignant.

An automatic method to classify stromal regions according to their maturity was studied by Reis et al. [113]. At Multiple scales, basic image features and Local Binary Patterns (LBP) were extracted. Random decision tree classifier was used to classify the stromal regions. A total of 55 H&E stained images of invasive breast carcinoma were used in the study. Classification accuracy of 84% was reported. A content-based histopathological image retrieval framework was proposed by Zheng et al. [114]. The classification and retrieval accuracy of 94% was reported. A combination of features to classify breast cancer tissues was studied in [115]. Curvelet and LBP features were extracted from the images. SVM, random forest, decision tree, and polynomial classifier were used for classification. A method to automatically detect and grade the lymphocytes in HER2+ BCHI was proposed by Basavanhally et al. [116]. Region growing and Markov random field algorithms were utilized for the detection and SVM used for classification. The proposed method was evaluated on a total of 41 images. Classification accuracy of 90% was reported.

The authors in [117] addressed the problem of grading invasive breast carcinoma using Grassmann manifold. A vector of locally aggregated descriptors encoding technique was designed. The classification accuracy of 95% was reported on their dataset and 91% on BreakHis dataset. Das et al. [118] proposed a dictionary-based approach for nuclear atypia scoring. The methods like sparse coding and dictionary learning algorithms were utilized for the automated grading of nuclear pleomorphism. The joint kernel-based supervised hashing approach was proposed by Jiang et al. [119]. The proposed approach integrates complementary features in a hashing framework. Classification accuracy of 91% within 16.5ms query time was reported.

Beck et al. [120] designed a model namely the C-Path system to measure the quantitative features from epithelial and stromal regions of breast cancer tissues. Standard and morphometric descriptors of image objects were considered. The features such as higher-level contextual features and global image features were considered. Baker et al. [121] proposed a framework for the classification of BCHI using a combination of K-means clustering and watershed algorithms for the segmentation. Morphology features were extracted from the segmented ROI. An accuracy of 70% and 86% were reported by rule-based and decision tree classifiers.

Irshad et al. [122] developed an automated mitotic detection framework based on different texture features. Texture features such as co-occurrence features, run-length features, and scale-invariant feature transform were extracted and used for the classification. A method for counting the mitotic cells in histopathological images was proposed in [123]. The intensity-based and Haralick features were extracted from the mitotic nuclei and surrounding stromal regions. Regenerative random forest classifier was utilized to classify the mitotic and non-mitotic nuclei. Nateghi et al. [124] reported a method to remove the non-mitotic cells from BCHI by using maximized intra-class weighted mean. The color, texture, and shape-based features were extracted. The classification of mitosis and non-mitosis was performed by using SVM with RBF kernel.

Classification using DL approach

Several studies have been carried out to classify BCHI into benign and malignant using DL techniques [125,126,127,128,129,130,131]. The features were extracted by using VGG16, VGG17, DenseNet, ResNet, InceptionV3, and AlexNet architectures and were classified using dense layers.

Fully Convolutional Neural Network (FCNN) architecture for detection and classification using 240 WSIs of the breast was proposed by Gece et al. [132]. DCNN architecture for classification of BCHI was developed by Burçak et al. [133]. A model was designed to compute weights in the network and update the parameters for faster backpropagation learning. A class structure-based DCNN architecture was developed by Han et al. [134] for multi- class classification of BCHI. An average accuracy of 93% was reported on BreakHis dataset. Nucleus guided feature extraction framework was proposed by Zheng et al. [135]. A CNN classifier for detecting the presence of invasive breast cancer from WSIs was proposed by Cruz-Roa et al. [136]. The model was trained using 400 images and validated on 200 cases from the Cancer Genome Atlas dataset.

BreakNet architecture for classification was developed by Togacar et al. [137]. The architecture is a combination of attention modules and hypercolumn techniques. The architecture consists of convolutional, dense, and residual blocks. The classification accuracy of 98% was reported on BreakHis dataset. Li et al. [138] proposed a DCNN model to address the issue of class variance and feature extraction from different magnification images. Xception model is used to extract the features. A residual learning-based CNN named ResHist for automatic diagnosis of BCHI was designed by Gour et al. [139]. The developed CNN architecture consists of 152 layers. The ResHist model achieved an accuracy of 84% and an F1-score of 90% for binary classification.

An approach for extracting the multilevel features by integrating CNN and RNN architecture was proposed by Yan et al. [140]. The model achieved an accuracy of 91% for four-class classification. Breast Cancer Histopathology Image Classification Network (BHCNet) was designed by Jiang et al. [141]. The architecture consists of a combination of residual modules and squeeze-and-excitation blocks to reduce overfitting. Classification accuracy ranging from 98-99% for binary classification and 90-93% for multi-class classification was reported. Transfer learning approach was utilized [142] for detection and classification of breast cancer. To extract local information from the images and to improve the classification accuracy, Interleaved DenseNet with SENet architecture was proposed by Li et al. [143]. Performance of the transfer learning techniques for classification of epithelial and stromal regions was investigated by Du et al. [144].

Wang et al. [145] proposed a hybrid structure for the classification of BCHI. The high-level feature was extracted by incorporating transfer learning and double-deep transfer learning techniques. The interactive approach was used to enhance the classification performance. The classification accuracy ranging from 96-98% was reported. Context-aware stacked CNN for analyzing a large contextual area in WSIs for classification was reported by Bejnordi et al. [146]. Inception_ResNet_V2 architecture [147] was used to classify BCHI. The pre-trained architectures namely ResNet50 and DenseNet-161 were used to extract the features and to detect IDC by Celik et al. [148]. The experiment was conducted on 277,524 image patches of 50X50 pixels. An accuracy of 91% was reported on DenseNet-161 and 94% on ResNet-50 respectively.

Sharma and Mehra [149] demonstrated the influence of layer-wise fine-tuning for the classification of images using a pertained network. The experiment was carried out using per-trained AlexNet architecture with 8 layers. From this study, the authors claimed that moderate level of fine-tuning is an ideal choice for classification. Alzubaidi et al. [150] utilized transfer learning technique to handle the inadequacy in training datasets. Hybrid DCNN architecture with parallel convolutional layers with multi-branch and residual links was designed. A selective attention mechanism was designed by Xu et al. [151] to identify potential regions from the BCHI. Haar wavelet-based spectral features were integrated with spatial features to reinforce the performance of CNN [152].

Authors in [153] implemented an ensemble of multiscale CNN architecture for the classification. The scaled images were used to fine-tune the pre-trained architectures namely DenseNet-161, ResNet-152, and ResNet-101. The authors also claimed that the proposed architecture reduces the time complexity with an accuracy of 91% on training datasets. Kausar et al. [154] proposed a method for image classification using CNN on wavelet decomposed images. An accuracy of 98% was reported on International Conference on Image Analysis and Recognition (ICIAR) datasets. Yang et al. [155] designed an architecture to focus the network only on specific regions of interest by incorporating a guided soft attention network. A patch-based classifier was developed by Roy et al. [156] by utilizing CNN for automation. The proposed method was tested on the ICIAR-2018 dataset. An accuracy ranging from 77-92% was reported.

To overcome the computational cost concerning very large images Nazeri et al. [157] proposed a two-stage CNN architecture. The “patch-wise” network is designed to extract the local information from the image patches. The “image-wise” network was designed to extract global information and to perform classification. An accuracy of 95% was reported on the validation sets. Bejnordi et al. [158] designed CNN architecture to identify and distinguish tumor-associated stromal regions in breast biopsies. A transfer learning approach with block-wise fine-tuning was utilized to learn the best features from the images to handle magnification dependent and magnification independent binary and eight class classification problems [159]. CNN-based approach was designed to classify images based on nuclear atypia grading [160]. Multi-scale feature concatenation for the classification of BCHI was carried out by Kausar et al. [161] using DCNN. Bi-directional Long-Short Term Memory model [162] approach was also proposed for the classification using context-based patch modeling. A deep transfer network [130] using Deep Convolutional Generative Adversarial Network (DCGAN) as a data augmentation technique was proposed to tackle the data imbalance problem.

Many studies have been carried out on the classification of mitotic and non-mitotic nuclei using DL techniques [163,164,165,166]. Multi-stage mitotic cell detection methods based on faster region CNN and deep CNNs was proposed by Mahmood et al. [167]. Resnet-50 network was utilized for feature extraction. Wu et al. [168] developed a fused FCNN architecture by combining the features from various layers to detect mitosis. The method was validated on the 2014 ICPR MITOSIS dataset. A two-phase CNN to reduce the class imbalance problem while classifying mitotic and non-mitotic nuclei was reported by [169].

Classification using hybrid approach

Various approaches have been proposed for the classification of BCHI using hybrid techniques [29, 170, 170] .Usually, automated feature engineering is used in the case of CNN. However, in a hybrid approach, handcrafted features are extracted and classified using neural network. CNN [29] architecture was designed to extract the features at different scales in BCHI. SVM classifier was used to classify the images. A sensitivity of 95% and an accuracy of 83% were reported. Wan et al. [170] reported a cascading ensemble approach for grading the BCHI. Multi-level features and semantic level features derived from CNN were extracted and trained an SVM classifier. The study was performed on 106 biopsy slides. Shallu et al. [171] utilized a pre-trained network namely VGG16, VGG19, and ResNet50 for feature extraction. An accuracy of 92% with logistic regression classifier for VGG16 was reported.

Comparison of two ML approaches for the classification was studied by [172]. Handcrafted features were extracted and trained on SVM and CNN classifier for classification. They reported an accuracy ranging from 96-98% for binary classification and 83-88% for multi-class classification. DNN models guided by the clustered algorithms to identify the hidden structural and statistical information from images were developed by Nahid et al. [173]. An accuracy of 91% on 200X and a precision of 96% on 40X images were reported. George et. al [174] proposed a nucleus-guided transfer learning approach for BCHI. The features were extracted using CNN pre-trained on ImageNet. SVM classifier was utilized to perform the classification of fused features. Outputs of the CNN and SVM were combined by using the belief theory-based classifier fusion technique. An accuracy of 96%, a sensitivity of 97%, and specificity of 96% were reported.

An analysis on cellularity estimation in BCHI was studied by Pei et al. [175]. The cellularity estimation was carried out by combining deep features, SVM, and tree boosting. The authors in [176] developed a predictive algorithm for the automation of benign and malignant proliferative breast lesions. The classifiers consist of two regression-based, two DL based, and two tree-based learning algorithms. The combination of a logistic regression model with active feature extraction outperformed other models with an accuracy of 91%.

A multi-network feature extraction model was developed by Wang et al. [177]. The features were extracted by utilizing four pre-trained DCNNs namely DenseNet-121, ResNet-50, Multi-level InceptionV3, and multi-level VGG-16. The relevant features were selected by the dual-network orthogonal low-rank learning technique. Classification was performed by using Ensemble_SVM classifier. The experiment was evaluated on ICIAR 2018 dataset and classification accuracy of 97% was reported. Incremental boosting CNN was designed [178] for classification. Global and local features were extracted by an ensemble of DCNNs from multi-scale images. The gradient boosting tree approach was utilized to perform the classification. Sharma and Mehra [179] explored and compared two ML techniques for the classification of BCHI. Handcrafted features namely Hu moment, color histogram, and Haralick textures were extracted and trained on conventional classifiers. In the second approach, pre-existing architectures were utilized to extract the features. Their study revealed that the use of transfer learning approaches for feature extraction gives better results when compared with handcrafted approaches. The authors in [180] explored ten pre-trained CNNs for feature extraction. The extracted features were classified using SVM classifiers. The feature extractor architectures namely ResNet 50, ResNet 101 and AlexNet with SVM classifiers were giving a better detection rate.

The cascading approach to detect mitotic nuclei was proposed by [181]. Handcrafted features such as morphology, intensity, and texture features along with CNN features were extracted and combined to detect the mitotic nuclei. The proposed method reported an F-measure of 0.73. Saha et al. [182] designed a DL architecture with handcrafted features to detect mitotic cells from BCHI. The architecture comprises five convolution layers, four max-pooling layers, four ReLU, and two fully connected layers. The morphology, texture, and intensity features were extracted. They reported a precision, recall, and F1-score of 92%, 88%, and 90% respectively. Beevi et al. [183] reported a method to detect mitosis using transfer learning technique. The features were extracted by combining a pre-trained CNN with a random trees classifier. An F1-score of 94% was reported. Mask-RCNN architecture was utilized to classify the mitotic and non-mitotic cells [184]. The handcrafted features namely shape and texture features are extracted from the image. A precision, recall, and F1-score of 93%, 81%, and 0.87 on the ICPR 2012 dataset was reported.

Figure 9 shows the illustration of various classification approaches used in the literature. We grouped them into traditional, DL, and hybrid approaches. In the “others” category we included K-NN, rule-based classifiers, boosting tree classifier, random subspace ensemble and belief theory-based classifiers. A summary of state-of-the-art approaches is given in Table 3.

Table 3 Summary of the state-of-the-art approaches

Full size table

Discussion

In recent years, BCHI analysis using medical image processing techniques has gained much popularity among the research community. Manual analysis of BCHI is prone to observer variability, human errors and is a tedious process. To mitigate these challenges, the use of CAD systems for the diagnosis of breast cancer using histopathological images is considered a potential alternative. However, there are challenges in developing CAD systems which are summarized below along with future directions.

Dataset

The development of CAD systems for analysis of BCHI is greatly dependent on the availability of large-scale datasets with annotations. Moreover, different annotations need to be provided for semantic segmentation and classification. For example, segmentation of nuclei needs annotations where nuclei regions are marked, image-level annotations are necessary for the image classification problem, bounding box based annotations are needed for detection problems. However, providing these annotations requires domain knowledge and is a tedious task. Considerable care must be taken while providing annotations since it has a great impact on the learning algorithm. The lack of a standard dataset in literature for the analysis of BCHI limits the development of CAD systems. In our view, the establishment of a standard large-scale dataset with annotations is the need of the hour as it provides a common platform to compare various algorithms. Summary of the most frequently used datasets in the literature is shown in Fig. 10. From the literature, it can be observed many research groups used BreakHis and Bioimaging 2015 database images for their study.

One of the most popular publicly available database for the analysis of BCHI is BreakHis [24]. From Figures 11 and 12 we can see that the number of cases for the benign class is significantly less as compared to the malignant class. This imbalance in classes results in poor training of DL models. The class imbalance problem is one of the challenges which needs to be addressed. A possible solution is to consider data augmentation to increase the number of samples for the benign class.

Magnification factors

To determine the condition of tumor, a pathologist analyses the images at varying magnifications such as 40x, 100x, 200x, and 400x. At 100x magnification, nuclei distribution and tissue structures are studied which helps in the determination of different types of cancer. At 400x magnification, various characteristics of nuclei are studied namely mitotic count, prominent nuclei and hyperchromatic nuclei. Hence, each magnification plays an important role in the determination of breast cancer. Despite this interest, most researchers have focused on developing CAD systems for 400x magnification images which may fail to capture the distribution of the tissue structure. It can be thus suggested that, in the future more attention could be given to the analysis of 100x and 200x magnification images for the determination of breast cancer.