Abstract

Introduction

Improvements in kidney transplantation have resulted in significant reductions in clinical acute rejection (AR) (8–14%) (1). Unfortunately, histological AR without evidence of kidney dysfunction (i.e. subclinical AR) occurs in >15% of protocol biopsies done within the first year (2–4). Without a protocol biopsy, patients with subclinical AR would be treated as excellent functioning transplants (TX). Moreover, 10-year allograft loss rates remain unacceptably high, 57% with deceased donor kidneys (5) and biopsy studies document significant rates of a progressive interstitial fibrosis and tubular atrophy in >50% of protocol biopsies starting as early as 1 year posttransplant (6–8).

Two factors contribute to AR: the failure to optimize immunosuppression and individual patient nonadherence (9,10). Currently, there is no validated test to measure or monitor the adequacy of immunosuppression, the failure of which is often first manifested as an AR episode. Subsequently, inadequate immunosuppression results in chronic rejection and allograft failure. The current standards for monitoring kidney transplant function are serum creatinine and estimated GFRs. Unfortunately, serum creatinine and eGFR are relatively insensitive markers requiring significant global injury before changing (3,11,12) and are influenced by multiple nonimmunological factors.

The gold standard for AR remains a kidney biopsy. Performing routine protocol biopsies is one strategy to diagnose and treat AR prior to extensive injury. A study of 28 patients 1 week posttransplant with stable creatinines showed that 21% had unsuspected “borderline” AR and 25% had inflammatory tubulitis (13). Other studies reveal a 29% prevalence of subclinical rejection (14) and that subclinical rejection with chronic allograft nephropathy was a risk factor for late graft loss (3). A study of 517 renal transplants followed after protocol biopsies showed that finding subclinical rejection significantly increased the risk of chronic rejection (15).

Limitations of biopsies include sampling errors, significant costs and patient risks. AR is a dynamic process and predicting rejection and managing immunosuppression require serial monitoring not possible using biopsies. Moreover, many patients present with acute dysfunction but no rejection is documented by biopsy (ADNR). Thus, there is a pressing need to develop a minimally invasive, objective metric for the diagnosis of AR and the adequacy of immunosuppression that can also identify ADNR.

We originally reported a peripheral blood gene expression signature by DNA microarrays to diagnose AR (16). Subsequently, others have reported quantitative polymerase chain reaction (qPCR) signatures of AR in peripheral blood based on genes selected from the literature or using microarrays (17–22). As the biomarker field has evolved, validation requires independently collected sample cohorts and avoidance of over-training during classifier discovery (23,24). Another limitation is that the currently published biomarkers are designed for two-way classifications, AR versus TX, when many biopsies reveal ADNR and that demands three-way classifiers.

We prospectively followed over 1000 kidney transplants from five different clinical centers (Transplant Genomics Collaborative Group) to identify 148 cases of unequivocal biopsy-proven AR (n = 63), ADNR (n = 39) and TX (n = 46). Global gene expression profiling was done on peripheral blood using DNA microarrays and robust three-way class prediction tools (25–27). Classifiers comprising the 200 highest value probesets ranked by the prediction accuracies with each tool were created with three different classifier tools to insure that our results were not subject to bias introduced by a single statistical method. Importantly, even using three different tools, the 200 highest value probeset classifiers identified were essentially the same. These 200 classifiers had a sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and area under the curve (AUC) for the validation cohort depending on the three different prediction tools used ranging from 82% to 100%, 76% to 95%, 76% to 95%, 79% to 100%, 84% to 100% and 0.817 to 0.968, respectively. Next, the Harrell bootstrapping method (28) based on sampling with replacement was used to demonstrate that these results, regardless of the tool used, were not the consequence of statistical over-fitting. Finally, to model the use of our test in real clinical practice, we developed a novel one-by-one prediction strategy in which we created a large reference set of 118 samples and then randomly took 10 samples each from the AR, ADNR and TX cohorts in the validation set. These were then blinded to phenotype and each sample was tested by itself against the entire reference set to model practice in a real clinical situation where there is only a single new patient sample obtained at any given time.

Materials and Methods

Gene expression profiling and statistical analysis

RNA was extracted from Paxgene tubes using the Paxgene Blood RNA system (PreAnalytiX GmbH, Hombrechtikon, Switzerland) and Ambion GLOBINclear (Life Technologies, Carlsbad, CA). Biotinylated cRNA was prepared with Ambion MessageAmp Biotin II kit (Ambion) and hybridized to Affymetrix Human Genome U133 Plus 2.0 GeneChips (Affymetrix Inc., Santa Clara, CA). Normalized signals were generated using frozen robust multi-array analysis (fRMA) in R (32,33). The complete strategy used to discover, refine and validate the biomarker panels is shown in Figure 1. Class predictions were performed with multiple tools: nearest centroids (NC), support vector machines (SVM) and diagonal linear discriminant analysis (DLDA). Predictive accuracy is calculated as true positives + true negatives/true positives + false positives + false negatives + true negatives. Other diagnostic metrics given are sensitivity, specificity, PPV, NPV and AUC. Receiver operating characteristic (ROC) curves were generated using pROC in R (34). Clinical study parameters were tested by multivariate logistic regression with an adjusted (Wald test) p-value and a local false discovery rate (FDR) calculation (q-value). Chi-square analysis was done using GraphPad (35). CEL files and normalized signal intensities are posted in NIH Gene Expression Omnibus http://www.ncbi.nlm.nih.gov/geo (accession number GSE15296).

Open in a separate window

Figure 1

Flow chart describing the cohort and bootstrapping strategies for biomarker discovery and validation.

Results

Three-way predictions

We randomly split the data from 148 samples into two cohorts, discovery and validation (Figure 1). Discovery was 32 AR, 20 ADNR and 23 TX, and validation was 32 AR, 19 ADNR and 22 TX. Normalization used fRMA (32,33). Probesets with median Log₂ signals less than 5.20 in ≥70% of samples were eliminated. A three-class univariate F-test was done on the discovery cohort (1000 random permutations, FDR <10%; BRB ArrayTools) yielding 2977 differentially expressed probesets (Table S1). The NC algorithm (25) was used to create a three-way classifier for AR, ADNR and TX in the discovery cohort revealing 200 high-value probesets defined by having the lowest class predictive error rates (Table 2; see also Supplemental Statistical Methods).

Table 2

Diagnostic metrics for the three-way nearest centroid classifiers for AR, ADNR and TX in discovery and validation cohorts

Method	Classifiers	% Predictive accuracy (discovery cohort)	% Predictive accuracy (validation cohort)	Sensitivity (%)	Specificity (%)	Positive predictive value (%)	Negative predictive value (%)	AUC	Sensitivity (%)	Specificity (%)	Positive predictive value (%)	Negative predictive value (%)	AUC
200	TX vs. AR	92	83	87	96	95	89	0.917	73	92	89	79	0.837
Classifiers	TX vs. ADNR	91	82	95	90	91	95	0.913	89	76	76	89	0.817
	AR vs. ADNR	92	90	87	100	100	86	0.933	89	92	89	92	0.893
100	TX vs. AR	91	83	87	93	91	90	0.903	76	88	84	82	0.825
Classifiers	TX vs. ADNR	98	81	95	100	100	95	0.975	84	79	80	83	0.814
	AR vs. ADNR	98	90	95	100	100	97	0.980	88	92	88	92	0.900
50	TX vs. AR	92	94	88	96	95	90	0.923	88	91	89	89	0.891
Classifiers	TX vs. ADNR	94	95	92	98	98	90	0.944	92	90	88	89	0.897
	AR vs. ADNR	97	93	95	97	100	97	0.969	89	91	89	89	0.893
25	TX vs. AR	89	92	81	96	95	84	0.890	88	90	90	89	0.894
Classifiers	TX vs. ADNR	95	95	95	95	95	95	0.948	92	92	89	89	0.898
	AR vs. ADNR	96	91	95	96	95	96	0.955	85	90	89	88	0.882

Open in a separate window

ADNR, acute dysfunction with no rejection by biopsy histology; AR, acute rejection; AUC, area under the curve; TX, excellent functioning transplant.

Thus, testing our locked classifier in the validation cohort demonstrated predictive accuracies of 83%, 82% and 90% for the TX versus AR, TX versus ADNR and AR versus ADNR, respectively (Table 2). The AUCs for the TX versus AR, the TX versus ADNR and the AR versus ADNR comparisons were 0.837, 0.817 and 0.893, respectively (Figure 2). The sensitivity, specificity, PPV and NPV for the three comparisons were in similar ranges and are shown in Table 2. To determine a possible minimum classifier set, we ranked the 200 probesets by p-values and tested the top 25, 50, 100 and 200 (Table 2). The conclusion is that given the highest value classifiers discovered using unbiased whole genome profiling, the total number of classifiers necessary for testing can be as little as 25. However, below that number the performance of our three-way classifier falls off dramatically to about 50% AUC at 10 or lower (data not shown).

Open in a separate window

Figure 2

Performance of the 200-probeset nearest centroids (NC) classifier discovered and locked in the discovery cohort tested on the validation cohort based on area under the curve (AUC).

Validation in one-by-one predictions

In clinical practice, the diagnostic value of a biomarker is challenged each time a single patient sample is acquired and analyzed. Thus, prediction strategies based on large cohorts of known clinical classifications do not address the performance of biomarkers in their intended application. Two problems exist with cohort-based analysis. First, signal normalization is typically done on the entire cohort, which is not the case in a clinical setting for one patient. Quantile normalization is a robust method but has two drawbacks: it cannot be used in clinical settings where samples must be processed individually or in small batches and data sets normalized separately are not comparable. fRMA overcomes these limitations by normalization of individual arrays to large publicly available microarray databases, allowing for estimates of probe-specific effects and variances to be precomputed and “frozen” (32,33). The second problem with cohort analysis is that all the clinical phenotypes are already known and classification is done on the entire cohort. To address these challenges, we removed 30 random samples from the validation cohort (10 AR, 10 ADNR, 10 TX), blinded their classifications and left a reference cohort of 118 samples with known phenotypes. Classification was done by adding one blinded sample at a time to the reference cohort. Using the 200 genes, three-way classifier derived in NC, we demonstrated an overall predictive accuracy of 80% and individual accuracies of 80% AR, 90% ADNR and 70% TX and AUCs of 0.885, 0.754 and 0.949 for the AR versus TX, the ADNR versus TX and the AR versus ADNR comparisons, respectively (Figure 3).

Open in a separate window

Figure 3

Performance of the 200-probeset nearest centroids (NC) classifier discovered and locked in the discovery cohort using a one-by-one validation on 30 randomly selected samples (10 AR, 10 ADNR and 10 TX) from the validation cohort based on area under the curve (AUC). ADNR, acute dysfunction with no rejection by biopsy histology; AR, acute rejection; TX, excellent functioning transplant.

Discussion

After 50 years of kidney transplantation, there is no validated molecular diagnostic for AR to obviate the necessity of a transplant biopsy in the face of acute transplant dysfunction. Ideally, molecular markers will serve as early warnings for immune-mediated injury, before renal function deteriorates, and also permit optimization of immunosuppression. We studied a total of 148 subjects with biopsy-proven phenotypes identified in five different clinical centers by following over 1000 transplant patients. Global RNA expression of peripheral blood was used to profile 63 patients with biopsy-proven AR, 39 patients with ADNR and 46 patients with excellent function and normal histology (TX).

We addressed several important and often overlooked aspects of biomarker discovery. To avoid over-training, we used a discovery cohort to establish the predictive equation and its corresponding classifiers, then locked these down and allowed no further modification. We then tested the diagnostic on our validation cohort. To demonstrate the robustness of our approach, we used multiple, publically available prediction tools to establish that our results are not simply tool-dependent artifacts. We used the bootstrapping method of Harrell to calculate optimism-corrected AUCs and demonstrated that our predictive accuracies are not inflated by over-fitting. We also modeled the actual clinical application of this diagnostic, with a new strategy optimized to normalizing individual samples by fRMA. We then used 30 blinded samples from the validation cohort and tested them one by one. Finally, we calculated the statistical power of our analysis (36) and determined that we have greater than 90% power at a significance level of p < 0.001. We conclude that peripheral blood gene expression profiling can be used to diagnose AR and ADNR in patients with acute kidney transplant dysfunction. An interesting finding is that we got the same results using the classic two-cohort strategy (discovery vs. validation) as we did using the entire sample set and creating our classifiers with the same tools but using the Harrell bootstrapping method to control for over-fitting. Thus, the current thinking that all biomarker signatures require independent validation cohorts may need to be reconsidered.

In the setting of acute kidney transplant dysfunction, we are the first to address the common clinical challenge of distinguishing AR from ADNR by using three-way instead of two-way classification algorithms. In the study of Li et al (18), a five-gene test was devised by qPCR from peripheral blood that effectively classified AR from a stable group (similar to our TX group). This two-way classifier was shown to have good specificity for AR against a set of non-AR biopsy-documented phenotypes comprising 12 borderline AR, 37 chronic allograft nephropathy, 16 calcineurin inhibitor (CNI) toxicities and 7 “other pathology.” Only their 16 CNI toxicities would be equivalent to our ADNR group, but if validated by testing on an independent cohort, this is an alternative strategy to use a two-way classifier to account for ADNR.

Two recent publications have described new urine signatures for AR. The first study with over 4000 urine specimens from 220 total patients describes a three-gene signature by qPCR that discriminated between biopsy specimens showing AR versus TX with AUCs of 0.83 in the validation cohort of 67 patients (37). Samples equivalent to our ADNR population were not tested but the signature was tested in cases with urinary tract, cytomegalovirus (CMV) and BKV infection. The second study involved urine profiling of six genes by qPCR and detection of CXCL9 and CXCL10 by protein ELISA in 280 adult and pediatric subjects as a two-way classifier of AR versus TX (38). Again, no clear ADNR group equivalent was studied but urinary detection of CXCL9 was revealed to have significant predictive accuracy. The strength of both these new studies was serial samples revealing the potential of these signatures to predict the future risk of presenting with clinical AR. We are currently doing a prospective serial monitoring study of blood and biopsies to validate the three-way classifiers presented here in 300 kidney transplants followed for 2 years.

The use of the Affymetrix microarray platform represents a well-established commercial technology, FDA-approved for diagnostic testing. New instrumentation enables automated, high-throughput analysis at the rate of >800 samples per week per instrument and a total undiscounted cost of approximately $275 per sample plus labor and indirect costs. Sampling of several million lymphocytes in the blood in each assay also eliminates the kind of sampling errors that are common to single biopsy cores.

We acknowledge limitations of this work that must be addressed in the next study. This is not a prospective, blinded study, and ultimately, validation of biomarkers for clinical use requires such a design. We had insufficient numbers of samples to test whether we could classify the different subtypes of T cell–mediated, histologically defined AR. Thus, we can diagnose AR, but biopsies will still be indicated when more detailed histological phenotyping is necessary. The majority of our patients were Caucasian. All the patients in this study were treated with CNIs and mycophenolic acid derivatives but the addition of steroids had no impact on the results (data not shown). Full confidence in our biomarkers will require validations in multiple new clinical centers to establish any race- and/or therapy-dependent differences, the impact of bacterial and viral infections and to remove any concern for the study of highly selected and phenotyped subjects.

Our population had no cases of pure ABMR. However, we had 12 mixed ABMR/T cell–mediated rejection cases but only 1 of the 12 was misclassified for AR. About 30% of our AR subjects had biopsies with positive C4d staining. However, supervised clustering to detect outliers did not indicate that our signatures were influenced by C4d status. At the time this study was done it was not common practice to measure donor-specific antibodies. However, we note the lack of correlation with C4d status for our data, recent data that ABMR can be present without C4d staining detected and the fact that the presence of DSA antibodies is not diagnostic of ABMR (30,31).

An open question remains what is the mechanism of ADNR since these patients were biopsied based on clinical judgments of suspected AR after efforts to exclude common causes of acute transplant dysfunction. While our results do not address this question, it is evident that renal transplant dysfunction is common to both AR and ADNR. The key point is that the levels of kidney dysfunction based on serum creatinines were not significantly different between AR and ADNR subjects. Thus, these gene expression differences are not based simply on renal function or renal injury. The second point is that our review of the biopsy histology for the ADNR patients simply revealed nonspecific and only focal tubular necrosis, interstitial edema, scattered foci of inflammatory cells that did not rise to even borderline AR and nonspecific arteriolar changes consistent but not diagnostic of CNI toxicity.

Finally, biopsy-based diagnosis is subject to the challenge of sampling errors and differences between the interpretations of individual pathologists (39). It is a fair question to ask whether the biopsy diagnoses are always correct and this goes to the underlying assumption in the field that the biopsy is the “gold standard.” To mitigate this limitation, we used the Banff schema classification and an independent central biopsy review of all samples to establish the phenotypes. Another question is how these signatures would reflect known causes of acute kidney transplant dysfunction (e.g. urinary tract infection, CMV and BK nephropathy). Our view is that there are already well-established, clinically validated and highly sensitive tests available to diagnose each of these. Thus, implementation and interpretation of our molecular diagnostic for AR and ADNR assume that clinicians would do this kind of laboratory testing in parallel. In complicated cases, a biopsy will still be required, though we note that a biopsy is also not definitive for sorting out AR versus BK.

There are several questions to answer in the next clinical study. First, can we use our molecular diagnostic to predict outcomes such as AR, especially diagnose subclinical AR, prior to enough tissue injury to result in kidney transplant dysfunction? Second, can we measure and ultimately optimize the adequacy of long-term immunosuppression by serial monitoring of blood gene expression? The design of the present study involved blood samples collected at the time of biopsies. Thus, we cannot claim at this point that our expression signatures are predictive of AR or ADNR. However, we would suggest that the absence of an AR gene profile in a patient sample would be a first measure of adequate immunosuppression and could be integrated into a serial blood monitoring protocol. Demonstrating the diagnosis of subclinical AR and the predictive capability of our classifiers would create the first objective measures of adequate immunosuppression. One potential value of our approach using global gene expression signatures developed by DNA microarrays rather than highly reduced qPCR signatures is that these more complicated predictive and immunosuppression adequacy signatures can be derived later from prospective studies like CTOT08. In turn, an objective metric for the real-time efficacy of immunosuppression will allow the individualization of drug therapy and enable the long-term serial monitoring necessary to optimize graft survival and minimize drug toxicity.

Supplementary Material

Acknowledgments

Abbreviations

Footnotes

References