Physicochemical properties-based hybrid machine learning technique for the prediction of SARS-CoV-2 T-cell epitopes as vaccine targets

Background

The SARS-CoV-2 virus underwent frequent genetic changes and mutations which caused widespread concerns all over the world (O’Toole & Hill, 2024). To guard against the emergence of these variations, there are two potential approaches: adjusting the formulation of current vaccines or creating an entirely new vaccine (Shang et al., 2020; Roper & Rehm, 2009). Given the urgency of the situation, PBVs stand out as a promising alternative. This option is advantageous due to its cost-effectiveness, shorter production timeline, safety, and potential to enhance both immunogenicity and cross-reactivity (Moss, 2022). Additionally, identifying SARS-CoV-2 epitopes using existing methods such as the NetMHC is often probabilistic, with limitations in predicting whether a peptide is an epitope or not (Nielsen et al., 2003). CTLpred can make deterministic predictions, but is restricted to peptides of length up to 9-mers only (Bhasin & Raghava, 2004). To address these limitations, this study introduces a hybrid ML model that can predict TCEs directly and of varying lengths (>9 amino acids).

Contributions

This research introduces several innovative contributions. Primarily, the study involves the development and evaluation of 27 hybrid ML techniques. These techniques are constructed through permutations and combinations of three classification algorithms, three feature weighting algorithms, and three feature selection techniques, all geared towards predicting TCEs of the SARS-CoV-2 virus. Second, we introduced a new feature extraction technique that extracts the physicochemical properties of peptides at the amino acid level. Third, we used heuristic and greedy search techniques to identify optimal features for training the models after extracting features from peptide sequences. Fourth, the focus of this study was achieving high accuracy in predicting TCEs and the proposed hybrid techniques showed promising results in terms of accuracy. The proposed models underwent evaluation based on various parameters such as area under the curve (AUC), sensitivity, specificity, Gini, F-score, and MCC. The results indicate that the combination of decision tree (DT) with chi-squared and forward search emerges as the most accurate and reliable predictive method for forecasting TCEs of SARS-CoV-2. Additionally, K-fold cross-validation (KFCV) was conducted, affirming the consistency and reliability of the proposed model for TCE prediction across all folds. The predicted epitopes have the potential to act as primary targets for designing a PBV.

The subsequent sections of the manuscript are organized as follows: The related work section presents a comprehensive literature review. The materials and methods section details the proposed methodology. The results section showcases the obtained results. Finally, the conclusion section concludes the manuscript.

Retrieval of peptide sequences

To obtain the peptide sequences for this study, we accessed the Immune Epitope Database (IEDB) repository (Vita et al., 2019). Specifically, we obtained two comma-separated value (CSV) files, wherein one of the files consisted of TCE sequences while the other file contained non-T-cell epitopes (NTCEs). All sequences are experimentally verified sequences and are of linear type. To perform binary classification, a target variable named “Class” was added to both CSV files, with epitope sequences.

Feature extraction

Each peptide sequence is characterized by its constituent amino acids and possesses physicochemical properties, which act as features in this study. To extract these features we performed feature extraction (FE) using the peptides (Osorio, Rondón-Villarreal & Torres, 2015) and peptider (Hofmann, Hare & GGobi Foundation, 2015) (Evaluation of Diversity in Nucleotide Libraries (R Package Peptider Version 0.2.2), 2015) packages of R programming language (R Core Team, 2013). Prior to applying FE, the duplicate peptide sequences were removed. The FE technique was employed separately to each CSV file (one containing TCE sequences and the other containing NTCE sequences) and a high-dimensional dataset consisting of 108 features for each sequence was produced. The two CSV files were merged into one CSV file. The merging of these CSV files facilitated the creation of a unified dataset ensuring that both TCEs and NTCEs labelled data instances are present in one single file. This comprehensive dataset enabled the training and evaluation of ML models on sequences belonging to both the classes i.e., T-cell epitope (TCE) sequences and non-T-cell epitope (NTCE) sequences. Table 1 enumerates the physicochemical properties employed in this investigation, and Table 2 provides a summary of the dataset post feature extraction (FE) procedures.

Feature selection

Feature selection (FS) constitutes a crucial phase in the machine learning (ML) process, wherein the objective is to choose the most pertinent features from a dataset for constructing a predictive model. The overarching aim of FS is to reduce the number of features in the dataset while preserving those that are most instrumental in enhancing the predictive accuracy of the model (Gupta & Rana, 2019). This process not only helps in reducing the computational cost and overfitting but also improves the generalization of the model. There are certain criteria that should be considered when selecting features. First, the selected features should be relevant to the problem being solved and should have a significant impact on the model’s performance. Second, the features should be independent of each other to avoid redundancy and multicollinearity, which can affect the stability and interpretability of the model. Ultimately, the chosen features should exhibit robustness against noise and outliers present in the data. Given the dataset’s high dimensionality, comprising 108 features, it becomes imperative to assign weights to all features and then identify the optimal subset. This process aims to augment the efficiency of the machine learning (ML) model. The subsequent subsections offer a detailed overview of the procedures involved in assigning feature weights and determining the optimal feature subset.

Feature weighting

Feature weighting is a technique in ML that involves assigning weights to the features to adjust their influence on the model’s output. The goal of FW is to give more importance to the relevant features and less importance to the irrelevant ones, thereby improving the accuracy and robustness of the model (Niño-Adan et al., 2021). In this study, three distinct FW techniques namely Information Gain (IG), Gain Ratio (GR) and Chi-Squared (ChS) from the FSelector package in R were employed to assign weights to the features (Romanski, Kotthoff & Schratz, 2023). A brief description of each technique is given as follows.

1. Information Gain: IG measures how much information is gained when a feature is used to split the data. Higher values indicate that a feature is more informative in distinguishing the classes. The function prototype of IG is given as information.gain(x, y, …). The function takes two mandatory parameters x and y, representing the feature and class variables respectively. The optional tuning parameters represented by three dots are N, the number of bins to discretize numeric variables, and FUN, the function to be used for discretization.

2. Gain Ratio: GR is an enhancement of IG that accounts for the number of possible splits a feature can have. It normalizes the IG by the intrinsic information of the feature, resulting in a more balanced measure. The function gain.ratio(x, y, …) takes the same parameters as IG

3. Chi-Squared: ChS tests the independence between a feature and the class variable. It measures the difference between the observed and expected frequencies of each class in each category of the feature. Higher values indicate that a feature is more related to the class variable. The function chi.squared(x, y …) takes the same parameters as IG.

IG was chosen for its ability to measure the information gained by a feature when used to split the data, making it particularly suitable for identifying informative features. GR, an enhancement of IG was selected to normalize the IG by the intrinsic information of the feature, providing a more balanced measure and reducing bias towards features with a large number of possible splits. ChS was deemed appropriate for testing the independence between a feature and the class variable, which is essential for identifying features that are significantly related to the target variable. It is important to note that all the function prototypes for these techniques take two mandatory parameters, x and y, representing the feature and class variables respectively, and some optional tuning parameters. IG and GR measure the amount of information gained by a feature, with GR normalizing the IG. ChS tests the independence between a feature and the class variable.

Finding the optimal subset of features

After assigning weights to the features, it is crucial to determine the optimal feature subset (OFS) for building an accurate and efficient ML model. The quest for the optimal feature subset entails the selection of a subset from a larger pool of available features. This subset is characterized by its ability to deliver the most effective predictive performance for the ML model (Kang & Kim, 2016). This procedure holds particular significance in high-dimensional datasets, where the abundance of features can be substantial and may detrimentally affect the model’s performance if not effectively managed. To obtain the optimal feature subset, three different effective techniques are employed in this study, namely hill-climbing search, forward search, and backward search (Więckowski, Kizielewicz & Kołodziejczyk, 2020). A brief explanation of each optimal feature searching techniques employed in this study is given next.

1. Hill climbing search (HcS): HcS is a heuristic optimization technique that iteratively evaluates the performance of different feature subsets and selects the subset that provides the highest accuracy. Starting with an initial feature subset, HcS evaluates the performance of all possible one-feature additions and selects the feature subset that provides the best improvement in accuracy. The process is repeated until no further improvements are possible.

2. Forward search (FwS): FwS represents a wrapper-based feature selection technique commencing with an empty feature subset. It systematically adds features in iterations, incorporating those that contribute the most significant accuracy improvement. FwS assesses the performance of all conceivable feature additions and ultimately chooses the feature subset yielding the most substantial enhancement in accuracy. This iterative process continues until no further improvements are attainable.

3. Backward search (BwS): BwS is similar to FwS except that it starts with the full feature subset and iteratively removes features that provide the least improvement in accuracy. BwS evaluates the performance of all possible feature deletions and selects the feature subset that provides the best improvement in accuracy. The process is repeated until no further improvements are possible

The outputs obtained from the three aforementioned optimal feature selection techniques are then inputted into three diverse ML algorithms, namely decision trees, neural networks, and random forests, which are further elaborated in the subsequent sub-section.

Machine learning classifiers used

The final hybrid techniques were formulated by incorporating three widely employed classification methods: decision tree (DT), random forest (RF) and neural network (NN). These techniques were integrated into the final classification step to facilitate a comparative study. Each method employs its intrinsic functionality for data classification, and the inclusion of these diverse classification techniques serves the purpose of utilizing three distinct approaches in the final phase of the proposed techniques. Typically, the DT algorithm employs a greedy approach for its classification task (Quinlan, 1986). DT partition the feature space into a set of disjoint regions, with each region corresponding to a unique class label. It then recursively splits the feature space based on feature values, aiming to maximize information gain or minimize impurity at each split point. On the other hand, the RF technique is an ensemble of decision trees, where multiple trees are generated, and the final classification is based on majority voting (Liaw & Wiener, 2002). Each tree in the forest is trained on a bootstrapped sample of the training data, and at each split point, a random subset of features is considered. Random forests mitigate overfitting and improve generalization performance by aggregating the predictions of multiple trees. NNs are a class of models inspired by the biological structure of the brain’s neural networks. They consist of interconnected layers of nodes (neurons) that process input data through a series of weighted connections and activation functions. The classification output of NN depends on the activation function used to build the network (Riedmiller & Braun, 1993). Table 3 provides an overview of the ML classifiers used along with their corresponding tuning parameters. The brief description of each classifier is provided in Supplemental Note S1.

Methodology

This section presents a summary of the methodology employed to create the proposed hybrid ML techniques. These models have been developed by combining three FW techniques, three OFS techniques, and three classification methods through permutation and combination approaches. Specifically, we employed three distinct FW techniques, namely Information Gain, Gain Ratio, and Chi-Squared, to assign weights to the features. These FW techniques were selected based on their ability to evaluate the relevance of features in distinguishing between different classes. Following the FW process, we employed three OFS techniques: hill-climbing search, forward search, and backward search, to determine the optimal feature subset. Each OFS technique iteratively evaluates different feature subsets to identify the subset that provides the best improvement in accuracy. Subsequently, the output from the FW and OFS techniques was integrated into three diverse ML algorithms: DT, RF, and NN. These ML algorithms were chosen for their ability to handle classification tasks and their distinct approaches to data classification. The workflow is presented in Fig. 1 which outlines the process. The proposed hybrid techniques consist of three stages and are outlined below.

Stage 1: During this phase, the peptide sequences of SARS-CoV-2 were acquired from IEDB in CSV format. Following this, feature extraction was executed on these sequences, utilizing the feature extraction techniques provided in the “peptides” and “peptider” packages of the R programming language. This process led to the generation of high-dimensional data, with 108 features extracted for each peptide sequence. Subsequently, the data produced during the feature extraction process from various CSV files were amalgamated into a unified CSV file. It is noteworthy that this phase constitutes a standardized procedure adopted by all hybrid techniques proposed in this study.

Stage 2: In Stage 2, several crucial steps were taken, encompassing the assignment of weights to extracted features and the determination of the optimal feature subset. Post FE, the subsequent phase involves assigning weights to each feature to gauge its relative importance. To accomplish this, three distinct FW techniques were employed. Following the FW process, the subsequent step encompassed determining the optimal feature subset, achieved through three different techniques. The output of each FW technique was separately fed into each OFS technique, and the resulting output from each combination was employed in Stage 3 of the research. Precisely, these outputs were utilized to train three distinct classification algorithms. The optimal features identified by various FS methods are outlined in Table S1. Equations (1) to (3) elucidate the model formulas, with the dependent variable “Class” and its corresponding independent variables for model training via Hill climbing, forward search, and backward search techniques, respectively.

(1) $$\mathrm{C}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}\sim \mathrm{f}\left(\mathrm{F}8\mathrm{\_}13,\mathrm{F}8\mathrm{\_}27,\dots \dots \dots \dots ,\mathrm{F}7\mathrm{\_}25,\mathrm{F}11\mathrm{\_}7\right)$$

(2) $$\mathrm{C}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}\sim \mathrm{f}\mathrm{F}8\mathrm{\_}47,\mathrm{F}8\mathrm{\_}4,\dots \dots \dots \dots ..,\mathrm{F}7\mathrm{\_}29,\mathrm{F}7\mathrm{\_}28)$$

(3) $$\mathrm{C}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}\sim \mathrm{f}\left(\mathrm{F}9\mathrm{\_}5,\mathrm{F}1,\dots \dots \dots \dots \dots ,\mathrm{F}7\mathrm{\_}31,\mathrm{F}8\mathrm{\_}22\right)$$

Equation 1 represents the model formula for training a predictive model using the HcS technique for feature selection. The dependent variable ‘Class’ is predicted based on a set of independent variables (features) denoted by F8_13, F8_27, …, F7_25, F11_7. Equation 2 represents the model formula for training a predictive model using the FwS technique for feature selection. Similar to Eq. (1), it predicts the ‘Class’ based on a specific subset of independent variables. Equation 3 represents the model formula for training a predictive model using the BwS technique for feature selection. Again, it predicts the ‘Class’ based on a subset of independent variables.

Stage 3: The initiation of Stage 3 involves partitioning the dataset, which contains the optimal features derived from each combination of FW and OFS techniques, into training and testing sets. The split ratio was configured at 80:20, allocating 80% of the dataset for model training and reserving the remaining 20% for predictions. Subsequent to training the models with the designated training datasets, their performance was validated using evaluation parameters detailed in the model evaluation section next. To ensure the reliability of the trained models, K-fold cross-validation was executed.

Area under the ROC curve

The ROC curve illustrates the relationship between the true positive rate (TPR) and the false positive rate (FPR) at different threshold settings (Bradley, 1997). The area under the ROC curve (AUROC) condenses this curve into a single value, serving as a comprehensive metric that summarizes the model’s efficacy in distinguishing between positive and negative classes.

K fold cross-validation

KFCV, or K-fold cross-validation, is a technique employed to assess the consistency and robustness of a model by dividing the original dataset into K subsets or folds of approximately equal size (Refaeilzadeh, Tang & Liu, 2009). The model undergoes training and evaluation K times, utilizing each fold once for evaluation while employing K-1 folds for training, as illustrated in Fig. 2. The performance metrics obtained from the K evaluations are then averaged, providing an estimate of the model’s generalization performance.

The selection of evaluation metrics in this study was based on their relevance to assessing the performance of predictive models for T-cell epitopes (TCEs) of the SARS-CoV-2 virus. In particular, the AUROC and MCC were chosen due to their effectiveness in evaluating binary classification models, particularly in the context of imbalanced datasets and the need for robust performance measures in biomedical applications. AUROC is a comprehensive metric that summarizes the model’s ability to discriminate between positive and negative classes across different threshold settings. Given the importance of accurately identifying TCEs, AUROC provides valuable insights into the overall discriminatory power of the predictive models. MCC, on the other hand, is a balanced measure that takes into account both true positives, true negatives, false positives, and false negatives, making it suitable for assessing the overall performance of binary classification models, especially in scenarios with imbalanced class distributions.

Discussion

This study addresses a critical need in the field of vaccine development against SARS-CoV-2 by proposing novel hybrid ML techniques for predicting TCEs. The significance of our research lies in its potential to accelerate the identification of vaccine targets, thereby contributing to the global efforts to combat COVID-19. By leveraging computational methods to predict TCEs, our study offers a cost-effective and time-efficient approach compared to traditional laboratory-based methods. Furthermore, our proposed techniques can predict peptides of variable lengths, including those longer than 9-mers, addressing a limitation of existing prediction techniques. Our findings not only advance the field of immunoinformatics but also hold promise for the development of PBVs against SARS-CoV-2. The deterministic nature of our predictive model distinguishes it from existing methods, which often rely on estimating binding potential rather than providing deterministic predictions. Moreover, the reliability and accuracy of our model, particularly when coupled with decision tree algorithms and feature selection techniques such as chi-squared and forward search, underscore its potential utility in vaccine design. In addition, the proposed hybrid ML models demonstrate robust performance across various evaluation metrics, underscoring their potential as valuable tools for accelerating the discovery and development of PBVs. Beyond COVID-19, the methodology presented in this study can be adapted and applied to other infectious diseases, providing researchers with a versatile approach for predicting TCEs and informing vaccine design efforts. The proposed work contributes to advancing the understanding of immune responses to viral pathogens and offers practical insights that could facilitate the development of next-generation vaccines with improved efficacy and specificity In nutshell, the study demonstrates the efficacy of hybrid ML approaches in predicting TCEs of SARS-CoV-2 and highlights their significance in expediting vaccine development. While further validation through experimental means i.e., in vivo and in vitro is warranted, our research lays the foundation for future investigations into advanced ML classifiers and additional physicochemical properties to enhance vaccine design strategies.

Glossary of Terms

To facilitate a better understanding of the biological concepts discussed in this manuscript, we provide brief definitions or summaries of key terms relevant to immunology, virology, and bioinformatics. These definitions aim to clarify fundamental concepts and terminology used throughout the study, ensuring that readers, both experts and non-experts in the field, can engage with the scientific content effectively. The following definitions are provided for terms frequently referenced in the context of our investigation: amino acid, peptide, epitope, T-cells, vaccine, peptide-based vaccines, SARS-CoV-2 and COVID-19.

Amino acid: Building blocks of proteins; characterized by an amino group, carboxyl group, and a variable side chain.

Peptide: Short chains of amino acids linked by peptide bonds; serve various biological functions including signalling and enzyme activity.

Epitope: Specific region on an antigen recognized by the immune system; crucial for antibody binding and immune response initiation.

T-cells: A type of white blood cell critical for cell-mediated immunity; involved in recognizing and destroying infected or abnormal cells.

Vaccine: Biological preparation that stimulates the immune system to produce immunity to a specific disease, typically by introducing weakened or killed pathogens or their antigens.

Peptide-based vaccines: Immunizations composed of specific peptide sequences derived from antigens, capable of inducing an immune response against pathogens or diseases.

SARS-CoV-2: A novel coronavirus responsible for the COVID-19 pandemic, characterized by its ability to cause severe respiratory illness in humans.

COVID-19: A highly contagious respiratory illness caused by the SARS-CoV-2 virus, characterized by symptoms ranging from mild to severe, including fever, cough, and difficulty breathing, with potential complications such as pneumonia and acute respiratory distress syndrome (ARDS).