Detection of sickle cell disease using deep neural networks and explainable artificial intelligence

1.1 Problems in identifying sickle cells and artificial intelligence (AI) as an aid

Being such a complex disorder, this is not easy to diagnose due to several reasons [16,17]. Some of the reasons are:

• Shape and size variability: Sickle cells can take on various shapes and sizes, making them difficult to differentiate from other red blood cells. The orientations also need to be considered while analysing.

• Low concentration: Sickle cells are found in low numbers in the blood, making detection challenging.

• Morphological variations: The form and appearance of sickle cells can alter over time, making identification complicated.

• Human error: Because sickle cell identification is frequently done manually by experienced professionals, it is vulnerable to human error and unpredictability.

• Lack of automation: Current methods for identifying sickle cells are time-consuming and labour-intensive, and automated solutions are needed to improve efficiency and accuracy.

In the field of healthcare, the use of AI had significantly enhanced the pace of progress. It has helped doctors/pathologists as an aid to diagnosis, decide treatment plans based on the patient data. It has also covered all the possible domains such as health management systems [18], telemetry and telemedicine [19,20], drug delivery data analysis, and precision medicine [21,22]. AI played a major role when human intervention became difficult.

When there are images that play a major role in diagnosis, deep learning is commonly used. A significant benefit of utilizing deep learning models is their capacity to automatically extract intricate patterns and high-level representations from data, often eliminating the need for manual feature engineering. It learns features, trains itself, and provides an output. Explainable AI techniques such as Grad-CAM and LIME can be used to explain the predictions made by the deep learning models. This allows humans to understand and interpret some conclusions made by the algorithm. It is essential in AI models as it characterizes model’s accuracy, fairness, transparency and outcomes. A few studies have used deep learning for SCD [23,24].

An unmet need In the diagnosis of sickle cells is the automated identification of sickle cells using deep learning and AI [25–28]. A study by de Haan et al. was on automated screening of sickle cells using a smartphone-based microscope. The framework used two different deep neural networks. These two networks have been designed to do specific tasks. One of them helped to standardize the image quality that has been captured from the experimented microscope to the laboratory-grade benchtop microscope. The other one took the output of the first network as its input and then performed operations for semantic segmentation between healthy and sickle cells within a blood smear. This method proved to be a more accurate and cost effective [29]. Using Hough Transform and some morphological tool can also enhance the deep learning models [30]. Alzubaidi et al. (2020) conducted a study on deep learning models for classifying red blood cells into circular, elongated, and other categories using lightweight deep learning models. To simplify the implementation of deep learning models and enhance their performance, transfer learning was employed. But it was important to note that transfer learning may not have a substantial impact on performance in medical image tasks when the source domain is entirely dissimilar from the target domain [26]. Following a similar approach, categorization into three datasets can be done for better results, one intended for domain transfer learning, the second to enhance robustness, and the third dataset for testing purposes. On following this approach, the accuracy obtained was 98.87% for the collected dataset [31]. Transfer learning with ResNet50 and DenseNet121 showed noticeable results regarding medical data classification [32,33]. Specifically, for sickle cells, an accuracy of 93.88% was found with ResNet50 [34]. Apart from sickle cell, deep learning has been widely applied in various medical tasks, showcasing its versatility and effectiveness in the healthcare domain such as detecting COVID-19 from chest images [35], automatic prediction and grading of retinopathy from retinal fundus images [36], wherein the models can not only predict the presence of retinopathy but also grade its severity, assisting healthcare professionals in timely intervention and management using three novel methods, and detecting glaucoma from fundus images, facilitating early intervention and preventing vision loss.

With different studies into consideration, major research gaps were found in sickle cell classification. There remain high chances of overlapping of cells or the three-dimensional geometry of a cell which may be overlooked, i.e. assumed to be normal from a different angle. Second, the generality of the model is restricted due to small and specific custom data. This resulted in reduction in accuracy when tested on new datasets. To tackle these issues, this research has been conducted. Explainable Artificial Intelligence (XAI) are being extensively used for demonstrating the trustworthiness of the models. However, very few studies have used XAI for sickle cell detection. Our main contributions are:

• Utilizing transfer learning for detection of sickle cell from the microscopic images of peripheral blood smear.

• Presenting comparative results and performance of three different models such as GoogLeNet, ResNet-18, and ResNet-50 for the detection of sickle cell samples.

• Further, XAI is used for demystification of the deep learning models. Grad-CAM has been used to make the predictions transparent and understandable.

The rest of the article is as follows: Section 2 discusses the methodology. Section 3 presents the results. Various discussions are made in Section 4. Section 5 includes a brief conclusion.

2.1 Dataset

Data have to be gathered before analysing or extracting any features from it. There are two different ways to proceed with this task. Data can be collected from medical facilities after obtaining prior ethical clearance. The second way is to use public datasets. Here we use a publically available open-source dataset from the University College of London. Data have been gathered to assess the effectiveness of automated image analysis algorithms in identifying sickle cells in blood samples from various digital images [37]. 1,985 images were captured that were semi supervised at 100× magnification with an objective lens of 1.4 NA, a camera capable of capturing coloured images and an X-Y stage that was motorized and employed for precise sample positioning. Of the 1,985 images, 740 had labels with sickle cells, while 1,134 were non-sickle cells. 111 images had no labels. Figure 4 shows both sickle and non-sickle images from the dataset used.

Figure 4

(a) Sickle cell image dataset and (b) other class dataset.

2.2 Dataset preparation

A dataset must be modified when obtained from an open-source repository. The raw dataset had folders with patient IDs. The downloaded file also included a text file which had the folder names and its corresponding binary value for true and false class. To segregate this faster, a python loop was incorporated which matched all the file names and then moved it to the desired location. The dataset was weakly supervised and a manual inspection/segregation was required. This reduced the final dataset to 1,664 images. This also aided in achieving a balance between the two classes. The size of the images in the dataset also needed adjustment. The images’ sizes were decreased by converting them from tiff to jpeg format to lessen the system’s processing requirements. This reduced the file size tremendously from 15 Megabytes to 550 kilobytes.

2.3 Network architecture

GoogLeNet, ResNet-18, and ResNet-50 are all popular convolutional neural network (CNN) architectures used for various computer vision tasks such as image classification. There have been several studies that have used these models in the field of digital pathology. While they share the common goal of extracting meaningful features from images, they differ in their architectural designs, depth, and complexity [38].

GoogLeNet introduced the concept of the Inception module, which employs multiple filters of different sizes within a single layer to capture diverse information. It aims to balance depth and computational efficiency by reducing the number of parameters. GoogLeNet’s efficiency and ability to handle variations in object sizes within images make it a valuable choice for tasks like medical image classification. The network contains nine Inception modules stacked with occasional max-pooling layers on top of each other for down sampling. This makes up a total of 22 layers which includes convolutional layers, max-pooling layers, and fully connected layers. The final layers consist of global average pooling, a fully connected layer, and the softmax output layer. Figure 5 shows the GoogLeNet architecture.

• Residual network (ResNet), is based on the concept of residual learning, which introduces skip connections or shortcuts to allow information to flow directly across layers [39]. This model is chosen for its relatively shallow architecture (18 layers). It strikes a balance between model complexity and computational efficiency, making it suitable for tasks where a lighter model is preferred, such as when dealing with limited computational resources or smaller datasets. The network consists of four residual blocks, each containing multiple convolutional layers with shortcut connections. ResNet-18 has 18 layers, including convolutional layers, batch normalization layers, max-pooling layers, and fully connected layers. The initial convolutional layer has a larger filter size (7 × 7) with stride 2 for down sampling, followed by max-pooling. The final layers include global average pooling, a fully connected layer, and the softmax output layer. Figure 6 shows the ResNet18 architecture.

Figure 5

GoogLeNet architecture.

Figure 6

ResNet18 architecture.

ResNet-50 is a deeper version of ResNet-18, with 50 layers in total. It follows the same residual learning principles as ResNet-18 but utilizes deeper residual blocks with more layers. This increased depth allows the model to capture more intricate features and patterns in the data, potentially improving its performance on more complex sickle cell classification tasks or datasets with greater variability. The network consists of five stages, each containing multiple residual blocks. The initial convolutional layer uses a smaller filter size (7 × 7) with stride 2 for down sampling, followed by max-pooling. The final layers are similar to ResNet-18, including global average pooling, a fully connected layer, and the softmax output layer. Figure 7 shows ResNet50 architecture [40].

Figure 7

ResNet50 architecture.

GoogLeNet introduced the idea of Inception modules to capture diverse features, while ResNet-18 and ResNet-50 introduced skip connections to enable deeper networks. ResNet-18 has 18 layers and four residual blocks, while ResNet-50 has 50 layers and five stages of residual blocks. ResNet-50 is deeper and more complex compared to both GoogLeNet and ResNet-18. The rationale behind choosing ResNet18, ResNet50, and GoogLeNet involves considering their architectural characteristics, transfer learning capabilities, community adoption and benchmark performance, interpretability features, availability in deep learning frameworks, and an empirical exploration approach. All three models (ResNet18, ResNet50, and GoogLeNet) have been pre-trained on large-scale datasets like ImageNet. ResNet architectures and GoogLeNet are widely adopted and have demonstrated strong performance in various image classification tasks. The community’s extensive use and benchmarking of these models provide confidence in their effectiveness. These contribute to the interpretability of these models. These factors collectively contribute to making informed choices for the sickle cell classification task.

2.4 Experimentation and model training

We acquired a diverse and representative dataset of sickle cell images from open source ensuring the dataset includes a sufficient number of samples with various conditions, resolutions, and lighting conditions. A microscope with 100× magnification, motorized XY stage and 1.4 NA objective lens has been used. Further, the dataset is to be split into 70, 20, and 10 ratio for training, validation, and testing, respectively. This study used transfer learning to exploit the knowledge obtained from a pre-trained CNN to classify sickle cells. Transfer learning is a powerful technique widely used in the field of deep learning. It involves leveraging knowledge gained from a pre-trained model, typically on a large dataset, to enhance the performance of a model on a different but related task. The aim of this study is to classify sickle cells using transfer learning with a pre-trained CNN. Matlab’s deep learning and machine learning package provided the computational framework for executing the transfer learning process. This comprehensive library facilitates seamless integration of deep learning methodologies, making it suitable for implementing complex neural network architectures and training procedures. The networks discussed in Section 2.3 are used here. These pre-trained networks are trained on large datasets such as ImageNet dataset and others. When the sickle cell dataset was introduced as input to the model, the weights and biases in the pre-trained network underwent a dynamic update process. This adjustment occurred according to the specific features present in the sickle cell dataset. Essentially, the pre-trained network served as a feature extractor or a knowledge source, enabling the model to adapt to the characteristics of the custom sickle cell dataset. To fine-tune the model for the task at hand, the early layers of the pre-trained model, which had learned generic features applicable to a wide range of images, were typically frozen. Freezing these layers prevents them from further updates during training. On the other hand, the later layers, including the convolutional layer and fully connected layer, known as the learnable layers, were replaced with task-specific layers. These layers were designated to capture features relevant to sickle cell classification. During the training process, the weights of these learnable layers were iteratively updated to better align with the characteristics of the sickle cell dataset. This dynamic updating allowed the model to learn task-specific features while still benefiting from the general knowledge encoded in the pre-trained layers. The hyperparameters are the one that control all these processes to get the best result by optimization. The parameters that can be used are maximum epoch, minimum batch size, initial learn rate, and optimizer.

2.5 Training

The training phase of deep learning is critical since models learn from data and modify their parameters. These are known as hyperparameters and govern the behaviour and performance of the learning algorithm and the model. Unlike the parameters of a model, which are learned through training, hyperparameters are manually set by the practitioner or determined through a search process. They define the structure and configuration of the learning algorithm and influence how the model learns and generalizes from the data [33].

The various hyperparameters compared in this study are: initial learn rate, minimum batch size, maximum epochs, and optimizer.

Initial learning rate represents the scale of the modification applied to the model’s parameters in each iteration of the optimization process. Additionally, it signifies the extent to which the model has absorbed information from the provided data. A too high learning rate may cause the model to overshoot the minimum, while a too low learning rate can slow down convergence or cause the model to get stuck in a local minimum. The preferred value for the classification task is 0.001. The second parameter, i.e. minimum batch size determines the number of training samples used in each training iteration. It affects the trade-off between computation efficiency and gradient accuracy. It depends on the availability of the GPU system. We perform trials on three different batch sizes: 32, 64, and 128. Larger batch sizes can provide computational efficiency, but smaller batches may offer better generalization. We compare these on different networks to know whether the batch size affects the performance. Maximum epochs is how often the complete dataset is fed through the model for training purposes. Too few epochs may result in underfitting, while too many epochs may lead to overfitting. We also try to compare the effect of epoch on the performance by keeping the other parameters constant. The model was configured to train utilizing the sgdm solver, which stands for Stochastic Gradient Descent with Momentum. This choice offers the benefit of hastened convergence, a smoother optimization path, and resilience to noise. It is particularly valuable in deep learning scenarios requiring gradient-based optimization.

The experimentation setup involved the utilization of a GPU system with CUDA Device (NVIDIA T400 4GB), resulting in a notable reduction in training time.

Each hyperparameter is analysed in terms of its impact on the training process, convergence, and model performance. All the three models were tried on the same hyperparameters and then compared based on their performance parameters. The performance parameters are calculated in Section 2.7.

2.6 Evaluation metrics

To evaluate the performance of the model for each class, we calculate the following parameters,

The ability of a model to identify true positives and true negatives correctly is termed as accuracy. The capability to accurately gauge the model’s recognition of positive instances is termed precision. Recall, on the other hand, evaluates the model’s competence in identifying all positive instances. Therefore, an effective model should exhibit high values for both of these metrics, much like an ideal situation. The F1 score provides a well-rounded compromise between precision and recall, signifying that there exists a commendable equilibrium between reducing false positives and false negatives by the model [41–43].

2.7 GRAD-CAM

In traditional AI models, such as deep learning neural networks, the decision-making process can be complex and opaque, making it difficult for humans to understand how and why a particular conclusion was reached. Hence, XAI techniques have been used for transparency and interpretability. In this study, we use Grad-CAM for the visualization [44–47]. It highlights the regions of an input image that contribute most to the model’s prediction for a particular class. Grad-CAM provides interpretability by generating heatmaps that visually highlight the regions of an image that are crucial for the model’s decision. This can help clinicians and researchers understand which parts of an image contribute to the classification of sickle cell, providing insights into the model’s decision-making process. The categorization of sickle cell may encompass nuanced features that are challenging for human observers to identify readily. Grad-CAM has the capability to emphasize these nuanced features, aiding in the revelation of patterns and traits within images that signify the presence of sickle cell. The challenges faced when using XAI with sickle cell is that the deep learning models used for image classification for sickle cell often have complex architectures. Explaining the decision-making process of intricate models can be challenging, and interpreting the significance of each parameter becomes more difficult as the complexity increases. Certain advanced machine learning models, especially deep neural networks, are often considered “black box” models. They make predictions based on complex interactions that are not easily interpretable. In spite of the challenges, Grad-CAM-produced heatmaps serve as a means of communication between data scientists and healthcare professionals. The visualizations may be more easily comprehensible for clinicians compared to the direct model outputs, promoting collaboration and facilitating the seamless integration of AI models into medical workflows.

3.1 XAI

XAI, when applied to the classification of SCD, allows us to discern the key regions in a blood smear image that significantly influences the model’s decision regarding whether the cells appear normal or sickle-shaped. This transparency is essential for instilling confidence in the model and ensuring its dependability for clinical applications. It operates by computing the gradients of a deep learning model’s output concerning the activation values in the last convolutional layer. These gradients signify the contribution of each pixel in the feature map to the ultimate classification decision. By weighting and superimposing these gradients onto the original image, Grad-CAM generates a heatmap, showcasing the specific regions that played a significant role in influencing the model’s prediction. Figures 13 and 14 show few of the Grad images for sickle cell and non-sickle image. The XAI technique was chosen for the best model (ResNet50 with 30 epochs and 128 batch size). It can be observed that, for regions near the presence of sickle-shaped cells, the heat map gives a higher value. The areas in the image where the heatmap has higher values are the regions that the network found most relevant for predicting the image as a sickle cell image. These are the parts of the image that likely contributed the most to the network’s decision, having those features which are common to all the sickle cell images in the dataset.

Figure 13

Sickle cell images under Grad-CAM.

Figure 14

Non sickle class images under Grad-CAM.

Similar to the classification of the other normal class of the dataset, it can be observed that the values of the heat map are high in almost 60–70% of the image. This is because for normal cells, there are no specific abnormality in the pattern that will be seen to consider it as a unique feature.