Abstract

INTRODUCTION

Despite recent advances, the benefits of immunotherapy are limited to a minority of patients with cancer. While thousands of clinical trials (1) have been underway to test novel approaches and combination strategies, only a few investigational regimens have shown added benefit and received regulatory approvals. A major impediment to furthering the success of immunotherapy is inadequate understanding of the complex interplay between tumor and immune system, and the diverse mechanisms of resistance to therapy in individual patients. Studies of tumor and tumor microenvironment (TME) have the potential to improve understanding of tumor biology and drug action or resistance, and to identify biomarkers of response and toxicities. Especially, analysis of clinical samples at baseline, on treatment, and at progression should be able to enhance knowledge of the mechanism of immuno-oncology agents, inform new drug development and combination strategies, and provide biomarker strategies (2,3). The integration of stringently validated biomarkers in immunotherapy trials could accelerate therapeutic development and optimization of clinical outcome.

Findings in early- and late-phase clinical trials have led to identification of candidate predictive markers for response to anti-PD-1/PD-L1 monotherapy, such as PD-L1 expression (4–6), CD8+ T-cell density (7), tumor mutational burden (TMB) (8,9), neoantigen prediction (10), transcriptomic profiles (11), T-cell receptor (TCR) clonality (7), and microsatellite instability (MSI) status (12). Biomarkers associated with poor outcomes have also been identified; examples include tumor loss of antigen presentation machinery (13,14), activation of Wnt/beta-catenin signaling (15), and cyclin-dependent kinase 5 (CDK5) expression that may dampen ability of T cells to reject tumors (16). Furthermore, biomarkers to predict immune-related adverse events (irAEs) are also of high interest, particularly for irAEs with life-threatening consequences.

However, to date different biomarkers have been investigated with variable success and overall show no significant therapy association with response from inhibition of checkpoint pathways. Determining the predictive accuracy of biomarkers for immunotherapy must involve a comprehensive approach that encompasses the complexity of tumor biology and the host immune system. While multi-omics technologies are widely available to support objectives of biomarker discovery, variability in assay methodology, assay data reporting, and specimen collection and processing procedures prevents comparison and interpretation of data across individual laboratories and clinical trials (17).

Elements critical to a systematic effort in biomarker development across different sites and studies must include: multidisciplinary expertise and capacity for complex tumor and immune profiling; assay platforms that not only are analytically validated but also demonstrate comparable assay performance across laboratories; appropriate clinical study design and sufficient sample sizes to make statistically valid conclusions; and a database for biomarker and clinical data integration with bioinformatics tools for correlative analysis within and across trials (Figure 1).

Open in a separate window

Figure 1.

Strategies to enhance the value of biomarker discovery through deep tumor-immune profiling on individual patients, biomarker analysis with clinical correlation within trials, and integrative analysis across trials.

METHODS

To facilitate the process of immune biomarker identification and comparison between different National Cancer Institute (NCI)-sponsored immunotherapy trials and laboratories, the NCI established the Cancer Immune Monitoring and Analysis Centers and Cancer Immunologic Data Commons (CIMAC-CIDC) Network (https://cimac-network.org/; “the Network”). The Network is composed of four multidisciplinary academic centers (the CIMACs) with capacity for state-of-the-art immune profiling assays, and a data coordination center (CIDC) that provides a database and informatics platforms for analysis and integration of clinical and biomarker data across trials.

The CIMAC-CIDC Network was launched in September 2017 through the Cancer Moonshot^SM initiative supported by the NCI. As the Network was being established, in parallel, it formed a collaboration with the Partnership for Accelerating Cancer Therapies (PACT), another Cancer Moonshot^SM project, and it became the public side of the public-private partnership (PPP) overseen by the Foundation for the National Institutes of Health (FNIH). This collaboration of the Network and PACT, launched in February 2018 (18), allowed for exchange of ideas between the industry partners from twelve leading biopharmaceutical companies, the Food and Drug Administration (FDA), NCI, and the academic partners in the Network. The Network and PACT agreed that validation and harmonization of biomarkers are essential for the future of immunotherapy development. In particular, the industry partners, through the PPP, provide financial support for the bioinformatics needs of the Network for optimization of data collection methodologies, data integration, and building a database of biomarker and clinical data at the CIDC. Additionally, they support development of novel biomarker assays and correlative studies in immuno-oncology clinical trials sponsored by NCI, industry, and other organizations such as academic centers.

This overview describes components required for the establishment of the Network. Also published in this volume are separate manuscripts that summarize the harmonization efforts by the Network on key assay platforms, including those for whole exome sequencing (WES) and RNA sequencing (RNAseq); mass cytometry by time of flight (CyTOF); and singleplex and multiplex immunohistochemistry/immunofluorescence (IHC/IF).

THE CIMAC-CIDC NETWORK INFRASTRUCTURE

The four CIMACs are located at Dana-Farber Cancer Institute, the Icahn School of Medicine at Mount Sinai, the University of Texas MD Anderson Cancer Center, and Stanford University. The CIDC is hosted at Dana-Farber Cancer Institute. Each CIMAC encompasses a multidisciplinary group of investigators with basic, translational, clinical, and computational expertise required for conducting complex correlative analyses.

The operational structure of the Network is depicted in Figure 2. Clinical trial teams of the NCI- and PACT-solicited trials seeking to collaborate with the Network apply through an established process that includes evaluation of scientific merit and feasibility of biomarker studies in the context of the clinical trial. For the selected trials, the CIMAC-CIDC investigators partner with the clinical trial team to design a biomarker plan and conduct immunoprofiling assays and correlative analyses. For correlative studies sponsored by the private sector PACT funds, the industry members also form a working group that helps to advise and refine the study design with the trial team. Blood and tissue specimens from the trials are collected at or transferred to designated central biorepositories for pathology quality control, processing, and distribution to the CIMACs. Guidelines and template agreements were developed for data access and sharing, specimen transfers, and intellectual property requirements (https://cimac-network.org/documents/).

Open in a separate window

Figure 2.

Organizational structure of the CIMAC-CIDC Network. Clinical trial teams supported by NCI or selected by PACT collaborate with the Network on the design and execution of correlative studies using specimens and data from clinical trials of immunotherapy. Assay data from CIMACs and clinical data from the trials are transferred to CIDC. CIMACs, trial investigators, and CIDC jointly perform integrative analyses of biomarker and clinical data using the CIDC bioinformatics platforms.

The four CIMACs provide a wide range of validated and harmonized assay platforms for comprehensive genomic, phenotypic, and functional characterizations for analysis of specimens from immunotherapy trials. Raw assay data generated by CIMAC laboratories are transferred to the CIDC. A set of clinical data elements is extracted from the clinical trial database and also transferred to the CIDC. CIDC functions as an immune-profiling data coordination center and facilitates Network activities through optimizing data collection methodologies, providing the central database investigator access, and ensuring bioinformatics tools for integrative data analysis of biomarker and clinical data, both within and across trials.

Currently over 30 clinical trials from various NCI trial networks, academic sites, and industry sponsors have been selected for collaboration with the CIMAC-CIDC Network. These trials range from phase I/pilot studies to randomized phase II and III trials and involve a variety of clinical settings, including pediatric malignancies, rare tumors, patients with preexisting autoimmune disorders, as well as patients with common solid tumors and hematological malignancies. Therapeutic strategies being tested in the trials include monotherapy with immunotherapy agents and combinations with other immune modulators, targeted agents, or chemotherapy/radiation. In addition to baseline tumor tissue and longitudinal blood sample collections, many early-phase trials also incorporate on-treatment and at-progression biopsies.

Overall, the Network has set out the following scientific and strategic goals:

• Conduct both unbiased hypothesis-driven and hypothesis-generating correlative studies in immuno-oncology trials, with the goal of identifying candidate biomarkers and integrating with clinical trial outcome;

• Establish the administrative infrastructure of the Network, including development of a CIMAC-CIDC Human Material Transfer Agreement (HMTA) involving multiple stakeholders (CIMAC-CIDC, NCI, clinical trial organizations and sponsors, and investigators), CIMAC-CIDC Guidelines to guide Network operations, and a study proposal intake process (documents found at https://cimac-network.org/documents/);

• Develop and implement a specimen collection and processing “umbrella” protocol supporting the immune profiling assays of the CIMACs, including standardization of pre-analytical conditions “(found at https://cimac-network.org/documents/);

• Analytically validate all Tier 1 and Tier 2 assays (Table 1) to be performed by CIMACs, conduct harmonization of the key assay platforms across different CIMACs, and establish reference standards for longitudinal monitoring of assay performance;

  Table 1.

  Tier 1, 2, and 3 assays in the CIMAC-CIDC Network

  Tier 1 assays (planned for all or most trials)	Tier 2 assays (planned for selected trials)	Tier 3 assays (highly novel and exploratory)
  • Whole Exome Sequencing (WES) • RNA sequencing (RNAseq) • The nCounter® platform for gene expression (Nanostring) • Multiplex immunohistochemistry/ immunofluorescence (mIHC/IF) • Singleplex IHC (sIHC) • Cytometry by Time Of Flight (CyTOF) • Olink immune-oncology immunoassay panel	• Circulating tumor DNA (ctDNA) • Grand Serology ELISA • Multiplexed ion beam imaging (MIBI) • CyTOF Phosphoflow • Single-cell TCR-sequencing (scTCR-seq) • TCR-seq • Assay for Transposase-Accessible Chromatin (ATACseq) • Microbiome analysis (16S sequencing, shotgun metagenomics)	• Single-cell RNA-sequencing (scRNAseq) • Cellular Indexing of Transcriptomes and Epitopes by Sequencing (CITE-Seq) • ELISPOT analysis for cytokine secreting cells • HLA tetramers for antigen-specific T cells

  Open in a separate window

• Support translational efforts in the immuno-oncology scientific community by providing access to a set of protocols for harmonized and validated assays that can be implemented by both academic and industrial laboratories;

• Establish the CIDC, including the bioinformatics platforms for within- and cross-trial analysis and integration of biomarker and clinical data;

• Facilitate broad data sharing with the larger research community by transferring data and findings from the CIDC to the NCI Cancer Research Data Commons (CRDC), including data from the industry sponsored trials;

• Establish an evidence base that can be used by academia and industry for clinical validation of assays and biomarkers for trials.

SELECTION AND PRIORITIZATION OF BIOMARKER MODULES FOR CLINICAL TESTING

The CIMACs selected a variety of platforms to provide comprehensive tumor and immune profiling for characterization of anti-tumor immune responses. These assays are categorized by tiers based on scientific priority and technical robustness for implementation in clinical trials (Table 1).

CONSIDERATIONS FOR SPECIMEN COLLECTION AND PRE-ANALYTICS

How specimens are collected and processed for preservation can have a large impact on the quality of assay data and correlative analyses. The CIMAC-CIDC Network includes over 30 clinical studies led by multiple different trial groups, across a projected collection of 40,000 specimens from about 3,400 patients. Allocation of limited specimen material for various assays needed for comprehensive profiling across CIMAC sites frequently poses a logistical challenge.

Efforts to achieve the goals set for the CIMAC-CIDC Network require that specimens meet a high standard of quality to ensure robust and comparable profiling. To guide investigators through sample distribution options among a variety of assays and provide standardized methods for specimen collection and handling, the Network and NCI developed the Specimen Collection “Umbrella” protocol (found at https://cimac-network.org/documents/). The Umbrella protocol addresses various steps in the “sample flow” from tissue or blood sample acquisition at trial sites, to immediate processing and storage at biorepositories, to subsequent processing and downstream distribution to the CIMAC laboratories. An overview of the Umbrella is provided in Table 2.

Table 2.

Pre-analytical elements in the CIMAC specimen collection “Umbrella” protocol

Specimen Type	Collection and Processing at Site	Immediate Processing at Biobank	Processing at Biobank for Distribution to CIMAC Labs	Intended Assay Use at CIMAC
Tissue biopsies
• De Novo Core Needle Biopsy • Endoscopic/ Punch Biopsy • De Novo Surgical Resection	• 1–2 cores or 1 segment (FFPE)	• Embed fixed tissue • Store blocks	• Unstained slides + H&E • DNA / RNA extraction	• Fresh Frozen Samples: • WES/germline, RNA-Seq, TCR-Seq • FFPE Samples: • IF, IHC, MIBI, WES/germline, RNA-Seq, TCR-Seq
	• 1–2 cores flash frozen or 1 segment flash frozen	• Store frozen	• DNA / RNA extraction
• Archival FFPE Material	• FFPE blocks or Unstained slides • Core punches	• Store blocks or Vacuum-seal slides • Refrigerate punches	• Unstained slides + H&E • DNA / RNA extraction
Blood
• Sodium Heparin Green-Top Tubes	• 30 mL Draw	• Isolate plasma and PBMCs • Smart tubes	• Ship Smart tube, plasma, or PBMCs • DNA (TCR-Seq)	• Plasma (Olink, ELISA) • PBMCs (CyTOF, TCR-Seq)
• Streck Cell-Free DNA Tubes	• 10 mL Draw	• Isolate plasma and Freeze aliquots	• Ship plasma aliquots	• cfDNA
• K2-EDTA Purple-Top Tubes	• 2 mL Draw (solid tumor germline) • 5–10 mL Draw (hematologic germline) • 2 mL Draw (TCR-Seq)	• Freeze germline aliquots • 2 mL aliquots (TCR-Seq)	• Extract and ship DNA aliquots	• Germline DNA, TCR-Seq
Bone marrow, CSF, stool
• Bone Marrow Aspirates • OR • Cerebrospinal Fluid	• Custom volume in K2-EDTA tubes	• Supernatant • Cell fraction	• Ship aliquots • Unstained slides + H&E • DNA / RNA extraction	• CyTOF, Olink, IF, IHC, MIBI, RNA-Seq
• Stool Samples	• Self-collection (Ship ambient or frozen)	• 2 mL aliquots (DNA-stabilizer) • Frozen stool	• Ship frozen aliquots	• 16S rRNA • Shotgun metagenomics

Open in a separate window

FFPE: Formalin-fixed paraffin-embedded

The Umbrella protocol was developed using an iterative approach. Optimizing performance of CIMAC assays and defining specimen pre-analytic requirements were instrumental in the assay validation and harmonization efforts of the CIMAC-CIDC, described later in this overview. Existing biorepository standard operating procedures (SOPs) that had been well validated with clinical trial samples were employed or adapted as far as possible. Where feasible, novel approaches were considered to support the need for flexibility and maximize use of limited tissue.

Since its development, the Umbrella protocol has been incorporated into several trials selected for collaboration with CIMAC-CIDC. Potentially, the Umbrella could have broader applicability beyond CIMAC-CIDC, as a consensus guidance for prospective immunotherapy trials to ensure high-quality specimen collections for downstream analysis.

PRINCIPLES FOR ASSAY VALIDATION AND HARMONIZATION

To enable robust and systematic biomarker analysis across the CIMACs and across clinical trials, objective quality control measures are required for all assays to be performed in the Network. These measures include each assay’s analytical validity, including its pre-specified level of variability and reproducibility, concordance between laboratories, as well as acceptance criteria appropriate to its intended use (Table 3).

Table 3.

Analytical performance metrics evaluated for CIMAC assays

• Analytes

• Technical platform(s)

• Reagents, controls, and calibrators

• Quality control parameters for specimens/analytes (e.g., cell viability, RNA/protein quality/integrity)

• Critical pre-analytic variables

• Analytical performance characteristics for each assay:

Current status and results of studies defining the sensitivity, specificity, accuracy, precision, reproducibility, reportable range, reference ranges/intervals (normal values), turnaround time, and failure rate of the assay.

Use of positive and negative controls, calibrators, and reference standards.

Number of samples in the reproducibility study.

How run-to-run variation (Coefficient of Variation; CV) was assessed and handled.

How inter-laboratory variability in the measurements was assessed and how these sources of variation were minimized to maintain performance at all sites within acceptable limits and to prevent drift or bias in the assay.

Scoring procedures and type of data to be acquired:

• Quantitative/continuously distributed

• Semi-quantitative/ordered categorical

• Qualitative/non-ordered categorical

Open in a separate window

Adapted from “Study Checklist for CTEP-Supported Early Phase Trials with Biomarker Assays”, found at http://ctep.cancer.gov/protocolDevelopment/ancillary_correlatives.htm

Assay harmonization for CIMAC-CIDC studies:

To allow comparisons of biomarker data across trials, concordance of these assays’ performance was established across the different CIMAC sites performing a given assay, following development of consensus protocols. To enable comparison and integration of assay data across different studies and sites, harmonization of laboratory-specific protocols and development of consensus SOPs are recommended. During this process, each participating laboratory evaluates and compares the validity of reagents, standards, methodologies, protocols, and data reporting specific to each laboratory. Development of consensus protocols enables data comparison and interpretation supporting biomarker development across different clinical trial sites (17) (Figure 3).

Open in a separate window

Figure 3.

Cross-site assay harmonization – An iterative process ensuring reproducibility and robustness of assays to overcome methodological and data variability across different sites. After Figure 1 in van der Burg SH et al. 2011 (17).

Across the CIMACs, the principles of harmonization have been applied and successfully completed for CIMAC Tier 1 assays assessing genomic (WES), transcriptomic (RNAseq), and phenotypic characterization of tumor (mIHC/mIF) and PBMC subtypes (CyTOF). Olink, also a Tier 1 assay, is validated and performed at a single CIMAC. The processes and results of the harmonization for individual assays are described in reports also published in this volume. All assays met the pre-defined acceptance criteria for concordance that had been established for each assay, demonstrating a high level of concordance of results between participating laboratories. The CIMAC assay SOPs are available on the CIMAC-CIDC website at https://cimac-network.org/assays/.

THE CANCER IMMUNOLOGIC DATA COMMONS (CIDC)

The Dana-Farber Cancer Institute maintains and hosts the CIDC for the CIMAC-CIDC Network. The CIDC provides bioinformatics methods and the computational expertise and resources to facilitate the analysis of immuno-oncology trial data for the Network (Figure 4). CIDC will receive clinical data from various sources, including NCI trial network trials, investigator-initiated trials, and industry trials for integrative analysis of the assay and clinical data. While the integrity of the provided data is maintained, the data will be mapped to a clinical data model and current standards. In conjunction with the CIMACs, CIDC has developed data standards and software for recording molecular, clinical, and metadata generated by the Network. As these data standards evolve, the CIDC will work with the NCI Center for Cancer Data Harmonization (CCDH) to ensure data is available for rapid sharing via the NCI CRDC as well as facilitating future cross-trial analyses.

Open in a separate window

Figure 4.

Network Data Coordination – CIDC provides the services and functionality below based on data received from CIMAC laboratories and Trial Organizations.

DISCUSSION

Assay harmonization was identified as an important objective for both the CIMAC-CIDC Network and the PACT PPP, to generate highly concordant and interpretable datasets across multiple laboratories and studies and to facilitate development of a database of biomarker and clinical data for secondary analyses. Harmonized assays reduce variability and enhance reproducibility of individual laboratory protocols and comparability of data across different laboratories and studies. In this regard, the primary objective of the Network activity has been achieved. In the second phase, the harmonized assays are being implemented for specimen analysis and correlation of assay data with clinical outcome variables. We hope the availability of CIMAC assay protocols and the publication of the data from CIMAC-CIDC Network harmonization projects will increase awareness in the immuno-oncology community of the importance of harmonization principles in successful biomarker identification, qualification, and implementation. It is the hope that these publicly available protocols will be adopted in academic and industry trials, allowing for uniform biomarker data generation enabling cross-trial analysis. Ultimately, the clinical utility of immune assays and optimization of immunotherapies based on biomarker data will depend on implementation of assay harmonization principles across the immuno-oncology community.

ACKNOWLEDGEMENTS

Footnotes

REFERENCES