Levels Propagation Approach to Image Segmentation: Application to Breast MR Images

Thresholding-Based Approaches

Al-Faris et al. [15] have proposed a method using a modified automatic seeded region growing based on a variation of the automated initial seed and threshold selection methodologies to segment breast tumors. The method was tested on images from the RIDER breast MRI dataset [39]. The proposed method suffers from a major drawback which consists of the use of a global threshold that is not sufficient to handle the internal heterogeneity characterizing some MRI breast tumors. Cui et al. [16] have proposed a method dedicated to malignant breast lesions segmentation. The introduced method performs a 3D segmentation by segmenting the volume slice by slice using a Gaussian mixture model to select thresholds and generate markers, followed by a watershed method to extract the lesion from the image. Unfortunately, the proposed method uses a complex technique to select thresholds and generate markers. On the other hand, it is based on the conventional watershed that is known to produce excessive over-segmentation which badly affects the accuracy of the segmentation.

Clustering-Based Approaches

Azmi et al. [17] have studied the performance of three clustering method categories called supervised, unsupervised, and semi-supervised on the RIDER breast tumor dataset. The supervised methods were the K-nearest neighbor algorithm (K-NN) [18], support vector machine (SVM) [19], and Bayesian classifier [20]. The unsupervised category was tested using the fuzzy C-means algorithm (FCM) [21]. And finally, the semi-supervised category included self training and improved self training (IMPST) [17]. The authors have found that the supervised and semi-supervised methods produce a satisfactory accuracy. However, these methods highly require initially labeled data. This fact constitutes one of the significant disadvantages of the methods. On the other hand, the unsupervised methods need no initial data but they produce actually a poor segmentation. Moftah et al. [22] have presented a segmentation method focused on identification of target objects on MRI breast images using an adaptation of the K-means clustering algorithm. The method is simple but it exhibits a loss of accuracy. Hassanien et al. [23] have proposed a segmentation and classification framework. The segmentation is performed by an adaptive ant-based clustering algorithm which is an improved version of the classical algorithm and the classification is done by the multilayer perceptron neural networks classifier. Unfortunately, the proposed framework suffers from a long searching time and a large amounts of calculations.

Graph-based approaches

McClymont et al. [24] have used mean-shift to produce an initial segmentation of the MR image. The resulting regions were next used to construct a region adjacency graph which was segmented by the graph-cuts partitioning algorithm. Yu et al. [25] have proposed a segmentation framework which incorporates mean-shift smoothing, superpixel-wise classification, pixel-wise graph-cuts partitioning, and morphological refinement. Graph-based approaches are easy to implement. However, the accuracy of such methods is highly related to the number of edges that are in the graph to segment. Specifically, the more edges to process, the more accurate segmentation to obtain. Hence, by considering the fact that the complexity of these methods increases with the increase of the number of the edges of the graph, it is obvious that a high accuracy requires a high complexity.

Other Approaches

Jayender et al. [26] have presented a method dedicated to breast invasive cancer segmentation. The idea behind their contribution is to model the underlying dynamics of the tumor by a linear dynamics system and use the system parameters to segment the cancer. Maicas et al. [27] have proposed a method based on a globally optimal inference in a continuous space using a shape prior computed from a semantic segmentation produced by a deep learning model. Despite the fact that the presented state-of-the-art methods are based on strong theoretical foundations, they actually generate a large amount of computations per final output.

Initialization Step

This phase serves to prepare the parameters and data to be processed in the next step. Here, a minor human intervention is needed. Indeed, according to the shape and location of the tumor in the image, the user selects the origin pixel. Since the location of the origin is important in the measure that it affects the distances between the origin and the pixels which belong to the boundary of the tumor, it is a key factor in determining the number of levels that are needed to process in order to reach the first level that is completely outside the tumor area. It is obvious that an optimal location of the origin would minimize that number of levels. Accordingly, such a location would be in fact the gravity center point of the tumor area. Thus, it is recommended to select it approximately around this point.

Segmentation Step

Just after the origin pixel is passed to LPA, the processing of levels starts. In this step, the levels are treated sequentially, starting from the closest level to the origin and propagating to upper levels until the stopping level is reached. Each level L_k, k ≥ 2 is processed with an associated threshold $T{h}_k^{\alpha }$ which is automatically obtained from the processing of its previous level L_k− 1, where α is a parameter to fix depending on the feature of the tumor in the MR image to analyze. Since the first level L₁ is too close to the origin which is supposedly near the gravity center of the tumor area, it is assumed that all its pixels are tumorous pixels. The remaining levels are processed one by one respecting the ascendent order of their distance to the origin. The processing of each level consists in differentiating its contaminated pixels (or in the process of being contaminated) from its safe pixels and this is done by analyzing all of its pixels with respect to the origin by starting from the top left corner pixel and pursuing the clockwise rotation. Specifically, the processing of the k-th level L_k, k ≥ 2 is done in two steps: in the first step, the distance in terms of intensity between every pixel $P_i^k,i=1,\mathrm{...},8k$ of the level L_k and the origin O is calculated using the following formula: $d(P_i^k,O)=\left|I\right(P_i^k)-I(O\left)\right|$, where 8k is the number of pixels of level L_k and $I\left(P_i^k\right)$ (resp. I(O)) is the intensity of the pixel $P_i^k$ (resp. O). In the second step, for every couple of neighboring pixels $P_i^k$ and $P_j^k$ of the level L_k, a test which consists in using the distances $d(P_i^k,O)$ and $d(P_j^k,O)$ (calculated in the first step) is performed to decide whether the two pixels are (in the process of being) contaminated or not. Thereby, if $d(P_i^k,O)\le T{h}_k^{\alpha }\vee d(P_j^k,O)\le T{h}_k^{\alpha }$, then it is considered that $P_i^k$ and $P_j^k$ are contaminated (or in the process of being contaminated), otherwise they are not. The associated threshold $T{h}_k^{\alpha }$ of the level L_k is obtained simply by using the greatest distance in term of intensity among the distances which are calculated for the contaminated pixels of the previous level L_k− 1. The exact formula is the following:

where $\mathcal{I}\left(L_{k-1}\right)=\left\{i\in \mathbb{N}:P_i^{k-1}\in L_{k-1}\right\}$, and α is a parameter related to the feature of the tumor in the MR image to analyze, such that

It is worth noting that the most important term in the formula (1) of the threshold $T{h}_k^{\alpha }$ is the term $\frac{1}{10}(e^{\frac{1}{k}\underset{i\in \mathcal{I}\left(L_k\right)}{max}\left\{d(P_i^{k-1},O)\right\}}-\alpha )$ which follows with enough fidelity the variation of the tumor through the different levels that characterize the LPA approach. However, the term $\frac{1}{10}\left(\underset{i\in \mathcal{I}\left(L_{k-1}\right)}{max}\left\{d(P_i^{k-1},O)\right\}\right)$ is added in the formula (1) to play a smoothing role in the threshold $T{h}_k^{\alpha }$ estimation during the progression in the processing of the levels of LPA. More details, illustration, and discussion on the threshold $T{h}_k^{\alpha }$ will be given in “Threshold Discussion.”

Once all the pairs of neighboring pixels of the level L_k are verified, the same processing is applied to the pixels of the level L_k+ 1 which is situated above the level L_k. The levels are treated one by one, producing therefore an organized propagation way that stops as soon as a specific level having a stopping condition is processed. Actually, it is worthwhile to note that the best level at which the propagation should stop has the feature that it is the closest level to the origin having all of its pixels outside the tumor area. Unfortunately, this situation is improbable and may not occur due to the noise that is present in the breast MR image. Specifically, the processing of a level that is completely outside the tumor area containing noisy pixels may result a small segmented component formed in each locality where a noisy pixel is present in that level. In this case, each noisy pixel erroneously generates a small segmented component composed of a successive supposed contaminated pixels. To overcome this issue, it is more convenient to stop the propagation at the first processed level which contains components of a tolerated size. For this purpose, the sum of pixels of each resulted component after the processing of each level is computed, and the propagation stops at the first level that produces components having at maximum three pixels. Note that this latter size is justified by the fact that the smallest size of a component generated by a noisy pixel with LPA is supposed to be equal to three.

Post-segmentation Step

Post-segmentation step is used to clean the image obtained from the previous step. In fact, the resulted segmented image does not only contain the tumor segment but also few disconnected small components around the component of interest which is in this case the tumor area. These components can be understood as tumorous regions. However, generally it is not the case. Actually, they are formed from the noise and speckles that are present in the MR image. Therefore, it is more significant to remove them to guarantee a good segmentation accuracy. In order to keep the tumor component only, a mathematical morphology opening operation is used to remove the unwanted small components. By using this operation, all the segments which have a sum of pixels that does not exceed a fix threshold are removed.

Evaluation Metrics

To evaluate the accuracy of the new approach LPA, a set of five metrics is used to compare the obtained segmentation results with their associated ground truths. Accordingly, two parameters should be found: the first parameter is the number of pixels that constitute the segmented region, while the second parameter is the number of pixels of the ground truth region that is delineated by the expert radiologists. These two parameters are used to calculate the following metrics: the true positive fraction (TPF), the true negative fraction (TNF), the sum of true volume fraction (STVF), the relative overlap (RO), and the misclassification rate (MCR). The formulas of the metrics can be found in [15, 17].

A good segmentation is indicated, on one side, by high values of TPF, TNF STVF, and RO and, on the other side, by a low value of MCR.

RIDER breast MRI Dataset: Results and Discussion

The first dataset on which the validation studies of the proposed approach are carried out is the Reference Image Database to Evaluate Therapy Response (RIDER) breast MRI dataset which belongs to the US National Cancer Institute. The dataset is publicly available for download from the Cancer Imaging Archive (TCIA) [39]. It contains coffee break repeat dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data of five subjects diagnosed with primary breast tumors. The collection includes also ground truth segmented images that have been manually identified by expert radiologists. All the ground truth images are 288 × 288 pixels and made for the axial slices of the breast MR images only.

For the evaluations on RIDER breast MRI dataset, LPA has been applied to forty randomly selected images from the dataset. The obtained overall segmentation results are compared to the ones of previous works. Specifically, six different segmentation methods were chosen for the purpose; five of them have been stated in the comparative study of Azmi et al. [17], namely K-Nearest Neighbor (K-NN), Support Vector Machine (SVM), Bayesian, Fuzzy C-Means (FCM), and Improved Self-Training (IMPST). Whilst, the sixth method is Modified Automatic Seeded Region Growing (MASRG) which is proposed by Al-Faris et al. [15]. The choice of these methods is motivated by the fact that all of them have been tested on the same dataset (RIDER breast MRI dataset) and evaluated by adopting the same evaluation metrics that were introduced in “Evaluation Metrics”. The experimental results are summarized in Table 2 and an example of the application of LPA to two images is shown in Fig. 9 in which Fig. 9a (resp. Fig. 9d) is an original breast MRI image that contains a tumor, Fig. 9b (resp. Fig. 9e) is the ground truth image that is delineated by expert radiologists from Fig. 9a (resp. Fig. 9d), and finally Fig. 9c (resp. Fig. 9f) is the extracted tumor from Fig. 9a (resp. Fig. 9d) using LPA. As it can be seen in Fig. 9, LPA has accurately extracted the tumor regions from the original images. Furthermore, the experimental results have shown that in general LPA has yielded competitive segmentation results to the concurrent state-of-the-art methods and has shown the potential to be successfully applied to breast tumor extraction from MR images. Specifically, LPA has had the best TPF mean score of all the evaluated methods, it has achieved a score of 0.90 which is by far higher than the one achieved by both MASRG and FCM. Furthermore, the RO mean score of LPA, which is 0.90, is significantly higher than the best RO mean of the remaining methods. LPA also has reached a MCR mean of 0.11 which is much less than the MCR mean of MASRG and FCM. Thus, LPA has had the best MCR mean of all the evaluated methods. The good results yielded by LPA are justified by the use of the novel thresholding technique that ensures a simple and accurate solution to overcome the issue of intensity inhomogeneity of breast tumors. Actually, the new thresholding technique proceeds by taking advantage from the organized propagation which characterizes LPA to progressively learn about the changes that occur in the intensity distribution within the tumor region. Therefore, it exploits the acquired knowledge to gradually refine the threshold value through the levels in order to deal with the intensity changes. In addition, LPA delimits the tumor area by examining at each time a pair neighboring pixels, this is more accurate than examining one by one pixels, because it is less sensitive to intensity alternations.

Table 2.

Segmentation accuracy results for the proposed approach and concurrent approaches for RIDER breast MRI dataset

	Statistics	TPF	TNF	STVF	RO	MCR
MASRG	Mean	0.82	0.90	1.73	0.75	0.18
Al-Faris et al. [15]	Std	0.10	0.10	0.10	0.09	0.10
	Max	0.99	0.99	1.87	0.88	0.48
	Min	0.52	0.65	1.46	0.49	0.01
IMPST	Mean	0.79	0.83	1.61	0.68	0.21
Azmi et al. [17]	Std	0.19	0.15	0.21	0.17	0.12
	Max	0.96	0.99	1.88	0.89	0.62
	Min	0.38	0.58	1.18	0.34	0.04
K-NN,	Mean	0.73	0.75	1.49	0.60	0.27
Fix and Hodges [18]	Std	0.13	0.15	0.19	0.12	0.10
	Max	0.87	0.97	1.75	0.76	0.58
	Min	0.42	0.99	1.85	0.36	0.13
SVM	Mean	0.75	0.71	1.46	0.60	0.25
Yao et al. [19]	Std	0.19	0.21	0.31	0.19	0.12
	Max	0.94	0.99	1.85	0.86	0.70
	Min	0.30	0.17	0.67	0.26	0.06
Bayesian	Mean	0.76	0.59	1.34	0.56	0.24
Wu et al. [20]	Std	0.19	0.25	0.35	0.17	0.08
	Max	0.95	0.97	1.76	0.77	0.70
	Min	0.30	0.09	0.69	0.22	0.05
FCM	Mean	0.82	0.42	1.24	0.59	0.18
Chen et al. [21]	Std	0.27	0.55	0.71	0.25	0.08
	Max	0.99	0.96	1.83	0.84	1.00
	Min	0	−0.73	−0.19	0	0.01
LPA,	Mean	0.90	0.79	1.70	0.90	0.11
The proposed approach	Std	0.04	0.12	0.13	0.38	0.13
	Max	0.97	0.96	1.89	1.89	0.85
	Min	0.77	0.49	1.32	0.51	0.02

Open in a new tab

However, LPA has yielded less favorable results regarding two evaluation metrics. Actually, it have produced the third-best TNF mean score and the second best STVF mean score of all the tested methods. This comes down to the fact that LPA slightly exceeds the border of a tumor, because it is based on the evaluation of pairs neighboring pixels. Consequently, the pixels which are adjacent to the tumor’s border pixels (contaminated pixels) are considered by LPA as being pixels that are in the process of being contaminated and are therefore added to the tumor area during the segmentation process. As a result, the region extracted by LPA exceeds the border that is delimited by expert radiologists in the ground truth image by one pixel all around the border of the tumor.

CHM-LIMED Breast MRI Dataset: Results and Discussion

The second dataset on which LPA is evaluated is the Chahids Mahmoudi Hospital-Laboratory of Medical Informatics (CMH-LIMED) breast MRI dataset, which contains fat-saturated T1-weighted images of eighteen patients having different types of tumors that vary in shape and location. The images of the dataset were collected in the radiology service of Chahids Mahmoudi Hospital (https://hcm-dz.com) in Tizi Ouzou, Algeria on September 2018, using a General Electric (GE) MRI device. Furthermore, all the medical imaging data of the dataset were totally anonymized before provision, and the dataset itself contains ground truth mask images that were made with the help of Dr Farid Kechih who is a confirmed expert radiologist and the head of the radiology service of the hospital. Both the original and ground truth images are 512 × 512 pixels. Moreover, it is noteworthy to mention that this study was approved by the head of the radiology service of the establishment.

In addition to what we have done previously, LPA has been tested on the CMH-LIMED breast MRI dataset and its results are compared against MASRG [15] which is region-based and the most recent method among the concurrent methods previously used for RIDER breast MRI dataset, as well as the golden standard Level set [40] that is a contour-based method.

However, unlike the first dataset, this second dataset is split into two sub-datasets, CMH-LIMED 1 and CMH-LIMED 2, according to two kinds of tumors, non-enhanced and enhanced tumors. More specifically, CMH-LIMED 1 is constituted of a set of twenty-four images from eight patients having non-enhanced tumors and CMH-LIMED 2 is constituted of a set of thirty images from ten patients with enhanced tumors. This would allow principally to have more precise and clear conclusions about the performance of the evaluated methods regarding the two kinds of tumors.

An example of the segmentation results of LPA and its concurrent methods for non-enhanced tumors is shown in Fig. 10. From Table 3 which reports the experimental results for CMH-LIMED 1, it is easy to deduce that LPA has produced encouraging results for non-enhanced tumors. In fact, it has achieved a high TPF mean score, around 0.86, which is far better than the TPF scores of the Level set and MASRG. In addition, it approximately yielded an RO mean score of 0.65 and an MCR mean score of 0.13 which are considered as the best scores among those of all the tested methods. However, LPA has achieved the second best mean scores in TFN and STVF metrics for which the best mean scores are obtained by level set. The worst scores for this sub-dataset are produced by MASRG. This is justified by the fact that the method uses the pixel having the highest intensity in the MRI image as a seed to initiate region growing. This technique is more suitable for enhanced tumors cases, because in such situations, the tumor region has the maximum mean intensity of all the regions of the MRI image. Unfortunately, considering the fact that after the injection of the contrast product, this latter is transported in the blood which passes through the heart inducing an enhancement of the corresponding region in the MRI image. Consequently, when the tumor region is not sufficiently enhanced, the seed pixel is chosen among the pixels of the heart region leading to the segmentation of this latter instead of the tumor region, as in the case of Fig. 10c.

Table 3.

Segmentation accuracy results for MASRG, Level set, and LPA for CMH-LIMED 1 breast MRI dataset

	Statistics	TPF	TNF	STVF	RO	MCR
MASRG	Mean	0.4622	−30.0527	−29.5905	0.0292	0.5395
Al-Faris et al. [15]	Std	0.4069	29.1071	29.1562	0.0388	0.4180
	Max	1.0000	−1.6749	−1.6749	0.1522	1.0000
	Min	0	−89.0161	−88.7972	0	0
Level set	Mean	0.7441	0.7706	1.5442	0.6369	0.2683
Li et al. [40]	Std	0.1675	0.2464	0.2749	0.1664	0.1731
	Max	0.9389	1.0000	1.8664	0.8754	0.6939
	Min	0.3061	0.2500	0.8397	0.2804	0.0611
LPA,	Mean	0.8632	0.6345	1.4977	0.6518	0.1368
the proposed approach	Std	0.1116	0.2498	0.2818	0.1388	0.1116
	Max	1.0000	0.9312	1.8387	0.8493	0.3878
	Min	0.6122	0.0469	0.9389	0.3822	0

Open in a new tab

Concerning the validation studies for enhanced tumors, the experimental results for CMH-LIMED 2 sub-dataset are outlined in Table 4. From the latter, one can observe that LPA has produced the best mean scores for all the evaluation metrics. Thus, it outperforms its concurrent state-of-the-art methods. Specifically, it has yielded a TPF mean of approximately 0.84, a TNF mean which is around 0.84, a STVF mean that is about 1.68, an RO mean of approximately 0.74, and, finally, an MCR mean approximating 0.15. Figure 11 illustrates a sample of segmentation results of an enhanced tumor by MASRG, Level set, as well as LPA.

Table 4.

Segmentation accuracy results for MASRG, Level set and LPA for CMH-LIMED 2 breast MRI dataset

	Statistics	TPF	TNF	STVF	RO	MCR
MASRG	Mean	0.7810	0.7655	1.5464	0.6611	0.2190
Al-Faris et al. [15]	Std	0.1701	0.4169	0.3655	0.1593	0.1701
	Max	0.9875	1.0000	1.8805	0.8829	0.6849
	Min	0.3151	−0.6526	0.3319	0.2774	0.0125
Level set	Mean	0.7105	0.8250	1.5295	0.6022	0.2796
Li et al. [40]	Std	0.2118	0.1747	0.2100	0.1713	0.2093
	Max	0.9917	1.0000	1.8318	0.8498	0.8178
	Min	0.1822	0.3435	1.0844	0.1743	0.0083
LPA,	Mean	0.8422	0.8445	1.6868	0.7443	0.1578
the proposed approach	Std	0.1337	0.2281	0.2356	0.1369	0.1337
	Max	0.9875	1.0000	1.9063	0.9108	0.5418
	Min	0.4582	0.0688	0.9168	0.3481	0.0125

Open in a new tab

Finally, to make a concluding statement about the overall performance of the proposed approach LPA, a summarization of the validation study for the whole CMH-LIMED breast MRI dataset is given in Table 5. In general, LPA has achieved a satisfying performance and exhibited the ability to be successfully applied to segment both non-enhanced and enhanced tumors as can be seen from Table 5. This is justified by the fact that LPA is outperformed by its concurrent methods in one evaluation metric only, namely TNF in which Level set has reached a mean score that is around 0.80 exceeding the TNF mean obtained by LPA with approximately 0.05. Nevertheless, the best mean scores for the remaining metrics are obtained by LPA.

Table 5.

Segmentation accuracy results for MASRG, Level set and LPA for CMH-LIMED breast MRI dataset

	Statistics	TPF	TNF	STVF	RO	MCR
MASRG	Mean	0.6535	−11.5618	−10.9083	0.4084	0.3433
Al-Faris et al. [15]	Std	0.3259	23.6899	23.8146	0.3367	0.3285
	Max	1.0000	1.0000	1.8805	0.8829	1.0000
	Min	0	−89.0161	−88.7972	0	0
Level set	Mean	0.7255	0.8026	1.5360	0.6176	0.2745
Li et al. [40]	Std	0.1924	0.2067	0.2387	0.1685	0.1924
	Max	0.9917	1.0000	1.8664	0.8754	0.8178
	Min	0.1822	0.2500	0.8397	0.1743	0.0083
LPA,	Mean	0.8515	0.7512	1.6027	0.7032	0.1485
the proposed approach	Std	0.1237	0.2581	0.2717	0.1441	0.1237
	Max	1.0000	1.0000	1.9063	0.9108	0.5418
	Min	0.4582	0.0469	0.9168	0.3481	0

Open in a new tab

Despite the advantages of LPA over the state-of-the-art methods and its yielded good results, it is worth to mention in this discussion that it has a limitation. Actually, it is true that the human intervention is minor when the MRI image to process presents one tumor. However, it gets higher with the increase in the number of tumors to segment in the same image.