Precision immunoprofiling by image analysis and artificial intelligence
- 1.9k Downloads
Clinical success of immunotherapy is driving the need for new prognostic and predictive assays to inform patient selection and stratification. This requirement can be met by a combination of computational pathology and artificial intelligence. Here, we critically assess computational approaches supporting the development of a standardized methodology in the assessment of immune-oncology biomarkers, such as PD-L1 and immune cell infiltrates. We examine immunoprofiling through spatial analysis of tumor-immune cell interactions and multiplexing technologies as a predictor of patient response to cancer treatment. Further, we discuss how integrated bioinformatics can enable the amalgamation of complex morphological phenotypes with the multiomics datasets that drive precision medicine. We provide an outline to machine learning (ML) and artificial intelligence tools and illustrate fields of application in immune-oncology, such as pattern-recognition in large and complex datasets and deep learning approaches for survival analysis. Synergies of surgical pathology and computational analyses are expected to improve patient stratification in immuno-oncology. We propose that future clinical demands will be best met by (1) dedicated research at the interface of pathology and bioinformatics, supported by professional societies, and (2) the integration of data sciences and digital image analysis in the professional education of pathologists.
KeywordsPersonalized medicine Immuno-oncology Immunotherapy Digital pathology Image analysis Machine learning Artificial intelligence
A picture is worth a thousand words. This is the essence of the technological transition from macropathology to micropathology as captured in Virchow’s core principle “omnis cellula et cellulae” first popularized in 1858 . One hundred sixty years later, pathologists are in a technological transition phase of similar importance : We are beginning to recognize that images of cells contain more information than what can be extracted by the human eye [20, 67, 83]. Computer-aided image analysis has the potential to make complex morphological information more accessible in daily diagnostic practice, improving prognostic and predictive patient stratification. Artificial intelligence has already been successfully employed in the setting of computational pathology to categorize diseases based on their molecular features [15, 21, 59]. In combination, it seems likely that image-based digital pathology in combination with artificial intelligence will become part of a pathologist’s tool repertoire in the near future.
Immuno-oncology requires a detailed understanding of the tumor microenvironment, including the identification and quantification of different immune cell subsets, their spatial context, and the expression of immune checkpoint markers. Changes in immune cell infiltration and biomarker expression before and after therapeutic intervention are critical parameters for clinical development . Image analysis tools can carry out complex and repetitive biomarker analyses with high precision and excellent reproducibility and can thus greatly assist pathologists in their key role to integrate clinical, morphologic, and molecular information for personalized treatment [20, 83, 84]. Actively engaging in image analysis methods is therefore becoming increasingly important for pathologists to maintain their role as leaders in precision medicine and diagnostics.
In recent decades, electronic data processing has transformed medical radiology, molecular diagnostics, and genetic testing. At the same time, the practice of pathology which forms a crucial link between these clinical disciplines has not changed significantly. The current convergence of new imaging technologies with tissue-based multiplexed immunohistochemistry (IHC) (reviewed in “Multiplexing”) and molecular phenotyping, our ability to digitize and process large collections of histology slides, and the promise of supporting human interpretation through automated analysis and artificial intelligence will have a dramatic impact on the field [15, 59].
While traditional medical device companies, such as Philips, GE, and Leica advance new platforms for digital pathology and commercialize new imaging technologies, the major IT companies, including Google, IBM, and Microsoft, as well as numerous start-up companies (e.g., PathAI) enter the space by applying their expertise in big data science and artificial intelligence to data analysis and integrated decision making. Pharmaceutical companies have also recognized the importance of advanced pathology to their own work. Roche has already received FDA clearance for the VENTANA MMR IHC Panel for patients diagnosed with colorectal cancer (CRC) . Developing and maintaining an understanding of digital imaging and data mining will therefore be beneficial skills for pathologists practicing in the twenty-first century [67, 83].
This paper provides a comprehensive outline of image analysis and machine learning (ML) applications for precision immunoprofiling. We will demonstrate how digital tools can facilitate pathology workflows in the assessment of established immune biomarkers and enable the deep characterization of the tumor microenvironment through spatial analysis and multiplexing. Further, we will identify computational methods driving morpho-molecular integration and deep learning methodologies for the discovery of novel therapeutic targets.
Computational pathology to assist in the assessment of established biomarkers
Immuno-oncology has been revolutionized by the introduction of immune checkpoint inhibitors (ICI). ICI are monoclonal antibodies targeting immuno-regulatory molecules on the surface of T cells, antigen-presenting cells, and neoplastic cell populations . Clinical success of reagents blocking the CTLA-4 (cytotoxic T lymphocyte-associated protein 4, CD152) and PD-1/PD-L1 checkpoints (programmed cell death protein 1, CD279; programmed death-ligand 1, CD274) has driven rapid regulatory approval for treatment of patients with both solid and hematologic malignancies . Assessment of PD-L1 expression by IHC has emerged as an important predictive biomarker for patients with non-small cell lung cancer (NSCLC) [12, 13], urothelial carcinoma , and renal cell cancer . However, assessment of PD-L1 is inherently difficult due to expression in both neoplastic and non-neoplastic cell populations, considerable marker heterogeneity and non-intuitive cutoffs. To complicate matters, PD-L1 positivity thresholds vary . Inter-observer variability of PD-L1 assessment by pathologists is a known problem [17, 77]. This may contribute to inaccurate patient stratification and the misinterpretation of the impact of PD-L1 expression on clinical outcome: Subjective decisions can lead to radically different therapeutic stratification when scoring around cutoffs.
Standardized biomarker assessment protocols are essential to accelerate clinical development of immuno-oncology therapeutics . Co-development of purpose-built computational pathology solutions in parallel to clinical trials may help to harmonize immunotherapy companion diagnostics, since image analysis algorithms can be easily shared and standardized across diagnostic labs. Alternatively, standardized biomarker testing may be conducted via telepathology at a central location. In addition, computational pathology applications can also increase the quality of biomarker assessment within an institution by using standardized tissue blocks and scanning procedures. While computational pathology will become a crucial tool in extracting quantitative information from digitized slides, digitization will also enable new applications. It will be possible to compare current cases with annotated phenotype libraries, hence effectively integrating a vast amount of knowledge into clinical decision-making . Taken together, computational pathology methods are likely to become important tools for making the assessment of immune biomarkers more reproducible, robust, and reliable.
Spatial analysis of tumor immune cell infiltration
The identification, localization, and spatial relationships of specific immune cells—before, during, or after therapy—have significant prognostic and predictive potential . Both commercial and freely available open-source image analysis solutions are available to perform area-based quantification of immune cells by IHC or immunofluorescence (IF) for prognostic and predictive analyses [6, 25, 54]. This includes the enumeration of peri- and intratumoral CD3+ and CD8+ T cell populations per mm2 of tumor tissue to form the Immunoscore® (IS) as marketed by HalioDx . This approach has shown independent prognostic value in CRC beyond usual risk factors and has strong potential to aid patient stratification in other solid tumors . A worldwide consortium-based validation study in stage I–III CRC reported significantly longer recurrence free intervals in patients with high numbers of tumor-infiltrating T cells . Recent data from stage III and stage IV CRC patients treated with adjuvant chemotherapy also indicate that the quantification of IS markers by image analysis has the potential to guide decisions on treatment duration and follow-up in advanced disease . However, the predictive potential of the Immunoscore® for immunotherapy response beyond well-known predictors, such as DNA mismatch-repair (MMR) deficiency remains to be investigated .
To empower more accurate patient stratification for immunotherapy, histological analysis should aim for a simultaneous characterization of both immune and tumor-related pathways in a single tissue sample [10, 29, 73]. Multiplexed immunoprofiling is key to generate comprehensive biomarker datasets for correlation with clinical parameters [11, 80]. Digital image analysis is an effective tool to extract comprehensive information on biomarker expression levels, co-localization, and compartmentalization. Technical approaches include virtual multistaining by computational methods, simultaneous multimarker staining of single slides, and sequential staining and quenching protocols.
Image registration is a powerful approach to generate virtual multistains from serial sections using computational methods: Following sectioning, staining, and scanning, serial sections can be computationally aligned by automated or user guided approaches . Digital image analysis is carried out and the results are used to generate virtual multimarker profiles. Standard manual or automated staining procedures can be used for visualization. Virtual multistaining also offers the unique opportunity of combining distinct histological methods, such as RNA in situ hybridization and IHC for multiparametric tissue profiling. This approach has been recently used to spatially correlate the expression of specific microRNAs, interferon-stimulated target genes, T cell infiltration, and the expression of cytotoxic effector molecules in the microenvironment of CRC . Limitations include the requirement for multiple tissue sections, inevitable sectioning artifacts, and the need to perform complex image transformation procedures for precise image alignment.
Highly optimized multistaining panels in combination with purpose-built digital image analysis software can overcome these drawbacks. However, substantial investment into a multimodal staining platform and analysis software is required to tap into this technology. Examples of this approach include the Vectra® Polaris™ Automated Quantitative Pathology Imaging System introduced by PerkinElmer . Using a single slide, this workflow allows to generate highly multiplexed IF panels to investigate six or more antigens in a strictly quantitative manner. Standardization is facilitated by commercially available marker panels, which may assist future clinical applications. A recent study by Mezheyeuski and colleagues elegantly illustrates the combination of the Vectra® approach with tissue microarray technology for a high-throughput profiling of the immune-environment in NSCLC samples . By combining a tumor-specific marker with a panel of immune-cell associated antigens, the authors analyze both quantitative and spatial information of specific lymphocyte subpopulations with RNA-expression levels and prognosis.
Interesting alternatives to simultaneous multiplexed IF are sequential workflows consistent of staining, digitalization, inactivation of fluorescent dyes, and re-staining. Profiling of up to 100 IF biomarkers on a single tissue section is technically feasible [29, 70]. Exemplary applications include multiplexed fluorescence microscopy method (MxIF) [29, 73] and multiepitope-ligand cartography (MELC) . Similar approaches have been tested and successfully applied for IHC protocols [66, 78]. The single channel outputs from each staining cycle can be easily merged into comprehensive expression maps for characterization of the tumor microenvironment using computational methods . However, individual optimization of the staining protocols, tissue degradation with iterative staining cycles, process time, and standardization may be hurdles in applying this technology to large clinical trial sample sets.
CO-Detection by indEXing (CODEX) is among the most recent and innovative approaches for high-dimensional imaging of antibody-tagged epitopes in FFPE tissue . CODEX staining uses antibodies labeled with unique DNA barcodes that are iteratively detected by in-situ polymerization with fluorescently labeled dNTP analogues. Tissue slides are stained in a single incubation step, followed by imaging cycles for the spatially resolved visualization of all antibody binding events. Pilot studies for have demonstrated the technical feasibility and power of this approach for deep profiling of immune tissue architecture . Application to immunoprofiling of cancer tissues is a logical next step for detailed analysis of the microenvironment in correlation with clinical and molecular parameters. A decisive advantage of this approach is the possibility to image CODEX-labels on a standard three-color fluorescence microscope or scanner.
Combining spatial imaging approaches with mass spectrometry may serve to reach yet another dimension in precision profiling of the tumor microenvironment. Multiplexed ion beam imaging (MIBI) uses antibodies labeled with isotopically pure elemental metal reporters (mass tags) to simultaneously detect up to 100 targets on FFPE tissue sections . Conventional histologic stains can be included in this panel, allowing precise reconstruction of virtual histology images with antibody expression data. Decisive technological advantages of MIBI include the stability of mass tag labels, a sensitivity exceeding chromogenic IHC by up to three log levels, and no spectral overlap between individual labels . Recent studies have demonstrated the technical feasibility to integrate RNAscope-based metal in situ hybridization with MIBI for multiparametric profiling . Purpose-built digital image analysis methods are available to reconstruct virtual histology images and explore cell phenotypes, spatial interaction, and morphological structures . Further technological advances using dissociated tissue samples allow to perform single cell proteomic analysis using a mass spectrometry-flow cytometry hybrid device, the so-called “CyTOF” . However, substantial investment and expertise in mass spectrometry are necessary to access this technology.
The recently proposed “Cancer Immunogram” suggests seven categories including tumor foreignness, general immune status, immune cell infiltration, absence of checkpoints, absence of soluble inhibitors, absence of inhibitory tumor metabolism, and tumor sensitivity to immune effectors as the most important predictors of immunotherapy response . As such, the “Cancer Immunogram” is highly integrative and includes both tumor- and immune-related parameters assessed with both molecular and image-based methods for individualized prediction of immunotherapy response. By providing continuous data on tissue-based parameters, such as immune cell infiltration and expression of immune checkpoints, computational pathology methods are ideally suited for data integration with molecular parameters.
Tumors with a high mutational load frequently exhibit an immunologically activated phenotype . Recent study has linked defects in DNA mismatch repair to high frequencies of neoantigens and immunotherapy response in advanced solid tumors [49, 62]. However, the correlation between genomic parameters and immunogenicity is far from linear . In some solid tumors, such as melanoma, a lack of association of neoantigen type and frequency with baseline immune activation has been elegantly demonstrated . In these tumors, strong regulatory T cell infiltration may curb the anti-tumoral immune response . Other known mechanisms that may impact tumor antigenicity are defects in antigen-presentation and overexpression of immunosuppressive molecules, such as CD47, PD-L1, and indolamine-2,3-dioxygenase [42, 43, 75]. These image-based features can be reliably captured by computational image analysis methods for in-depth profiling of the tumor microenvironment.
Incorporating digital information on immune cell infiltration with molecular data is a powerful approach to inform the “Cancer Immunogram.” Conde and colleagues recently provided a proof of principle for the association of specific genetic alterations with defined immunophenotypes by combining digital assessment of CD8+ T cell infiltration in lung squamous-cell carcinoma with manual scoring of PD-L1 and targeted next generation sequencing . Joining digital and molecular pathology also facilitates the development of novel assays to predict immunotherapy response. We have recently developed and validated a computational pathology assay that identifies specific PD1-positive subpopulations in NSCLC as a powerful predictive indicator of response to ICI treatment . Methodologically, this assay was driven by morpho-molecular integration of RNA sequencing data with tissue-based methods. Computational image analysis was a key tool in this process, driving the development of a standardized algorithm for the detection of tumor-infiltrating lymphocyte populations with uniquely high expression levels of PD1 protein (PD1T lymphocytes). PD1T cells were reproducibly detected and quantified in pretreatment biopsies of lung cancer patients and correlated strongly with treatment response to ICI in two independent clinical cohorts. This translational approach highlights how digital image analysis can represent a powerful companion diagnostic for cancer immunotherapy applications.
Machine learning and artificial intelligence
ML will transform the field of immuno-oncology. We expect that ML will drive a paradigm shift in the way data are collected and analyzed to discover new prognostic markers or to construct more rigorous risk classification to empower stratified medicine. ML is essentially a set of computer algorithms that learn generic rules to perform any given task directly from data, without requirements of predefined knowledge or domain expertise. This, in many ways, is similar to how pathologists have previously gained knowledge and expertise through continuous practice that has led to new diagnostic classifications or prognostic factors in the clinical routine.
Since ML is heavily data-driven, it enables a means to derive unbiased statistics from data. Accurate and continuous variables are more informative and can provide more biologically relevant information than the semi-quantitative scores presently implemented in diagnostic practice. Although several classical regression models exist to facilitate the discovery process of predictive or prognostic factors, the power of ML surpasses that of traditional tools when data is large and complex. ML can crunch through a vast amount of data and learn complex relationships between parameters and outcomes without the need to prespecify the relationships as normally required in traditional regression models. Nonetheless, data-driven approaches have critical downsides if not used with caution. ML can learn meaningless, biologically uninterpretable features that correlate to outcomes if the data has not been carefully preprocessed to remove any spurious features. Training datasets must have been generated in a highly standardized manner; otherwise, AI algorithms are likely to “misinterpret” sampling differences and artifacts between sets as distinctive biological characteristics. This phenomenon is known as “garbage in, garbage out.” Expert pathologist input in the training of image analysis algorithms is therefore of critical importance. Furthermore, ML tends to “overfit” to discovery data which results in an over-optimistic estimate of the performance of a model, while, in fact, it cannot be generalized well to new unseen datasets. It is, therefore, necessary that ML models are rigorously validated on new datasets that are independent of those used during the model development. Originating from the field of computer vision, deep convolutional neural network (DCNN) is a ML approach that is specialized in image analysis tasks [46, 50, 51]. In recent years, DCNN has been successfully applied to key applications in diagnostic pathology, including cell classification , cell enumeration , tumor grading , cancer diagnosis , and cancer prognostication .
Application of machine learning in immuno-oncology: pattern recognition
In immuno-oncology, ML as a pattern recognition tool enables an accurate and reproducible means for the unbiased assessment of regularities in the expression of immunohistochemical markers, tumor morphology, and the spatial distribution of tumor infiltrating lymphocytes (TILs). The ability of ML tools to detect key features in complex immunophenotypic datasets underlines their potential importance for the development of novel predictive models in cancer research. Initial studies underline the potential of ML methods for clinical translation:
Yuan et al. computationally profiled cells of triple-negative breast cancer cohorts on H&E-stained sections and were able to uncover three categories of lymphocytes (intra-tumor, adjacent-tumor, and distal-tumor) with an unsupervised clustering method based on the proximities of lymphocytes to tumor cells . Interestingly, the ratio between the total number of intratumoural lymphocytes and the number of cancer cells was strongly associated with disease-specific survival and strongly correlated with the expression of CTLA-4, a known immunotherapy target. Saltz et al. employed a DCNN model to determine the probability of TIL infiltration for every small region of H&E stained sections . This resulted in a map which indicates the degree of local lymphocyte infiltration. Unsupervised clustering methods were employed to group small TIL regions into spatially coherent structures . The resulting clustering patterns were characterized using parameters related to cluster size and shape. Importantly, the covariates summarizing complex characteristics of clusters were found to be associated with overall survival in various cancer groups, illustrating the potential of ML methodologies applied to standard H&E slides for the development of biomarkers in immuno-oncology.
Heindl et al. demonstrated that quantifying the number of spatial clusters or hotspots of immune cells and cancer cells is prognostic in ER+ breast cancer using a fully automated H&E stained image analysis algorithm . The spatial clustering is identified as an area in which the number of cells of interest is greater than expected by chance given the distribution of the cells on the whole tissue section. The increased immune spatial clustering is associated with poor prognosis and has a level of prognostic significance on par with the IHC test  and the Oncotype DX 21-gene recurrence score . Interestingly, immune scores which are related to the ratios of distinct categories of lymphocytes to the total number of cancer cells did not provide prognostic information in this study. This illustrates the importance of ML tools to detect novel morphologic features that provide prognostic information beyond established classifications.
Application of machine learning in Immuno-oncology: survival analysis
In medical research, survival analysis is a traditional means to assess the prognostic significance of each candidate covariate. The most widely used survival model is the Cox proportional hazards model. It assumes that the risk of an event of interest to occur (failure) is not time-dependent, is determined by a linear combination of all covariates, and might also include interaction terms between the covariates. This somewhat too simplistic assumption may not be able to capture the precise effects of the observed medical covariates on the risk of failure. This is particularly true in immuno-oncology where variable effects of immune infiltrates and immune-related gene signatures are observed depending on the tumor type and host immune status . DNN applied to survival analysis could provide a better way to model complex nonlinear relationships among prognostic factors that better fit to the survival data.
It has been shown that DNN performs better than the Cox proportional hazard model in various medical applications [24, 58, 86]. Katzman et al. investigated the risk associated with treatment choices and demonstrated that their DeepSurv system could provide treatment recommendations that increase the median survival time of patients . Yousefi et al. used their SurvivalNet to analyze large-scale genomic profiles obtained from the TCGA database [14, 86]. They have demonstrated the robustness of the model across different cancer types even if the number of input covariates is sometimes considerably larger than the sample size. Moreover, the model also allows the interpretation of the prognostic significance of individual covariates based on their contributions to overall risk. The same model has been extended to allow integration between histology image and genomic data in a unified framework . Likewise, specific methylome signatures queried by ML were shown to be suitable for prediction of response to immune-checkpoint inhibitors. In analogy to image analysis, this modality takes both neoplastic and reactive cells into account . These results suggest the potential of deep learning as a discovery tool to provide insight into the biology of immuno-oncology.
Regulatory issues in the use of digital pathology devices have been broadly recognized by national and international bodies. Pioneering companies have recently received CE certification for routine pathology applications in the European Union under the In vitro diagnostic medical devices directive . Comprehensive, non-binding recommendations have been issued by the Food and Drug Administration (FDA) and the College of American pathologists (CAP) [3, 65]. These guidelines highlight two important aspects of testing emerging biomarkers for immunotherapy by computational pathology methods: First, the quality and reliability of the imaging system as recommended by the FDA and mandated by the CE certification process, and second, the consistency of diagnoses made by pathologists using digital systems as advised by the CAP [3, 65]. Both are of essential importance in the design of reliable computational pathology workflows to inform clinical decision making.
Concerns have been raised that inaccurate study design, statistical analysis, and reporting of research lead to a significant waste of research funding and inaccurate scientific conclusions . A major challenge inherent to the use of digital image analysis is the substantial expansion in the number of variables. Although it may seem attractive from a scientific perspective to extract as much information as possible from a limited amount of tissue, it is important to recognize that big data mining inherently increases the level of statistical noise. Robust statistical considerations are required to avoid multiple testing problems and misleading conclusions. As with any diagnostic tool, digital image analysis is therefore critically dependent on scientific rigor, reliable documentation, consequent quality control, and adequate training of the pathology workforce . Unsolved problems also concern the standardization of the preanalytical steps before the imaging and analysis of tissue slides. A perfect example is Ki-67 staining, which has proven difficult to harmonize between labs despite being one of the most frequently performed assay of its kind . The same applies to the majority of immunohistochemical stains, in particular those where expression levels matter.
Advanced multiparametric imaging applications and ML have the potential to translate our evolving understanding of tumor-host interaction into better patient stratification and new treatment strategies. Computational pathology will help to derive complete, standardized, and reproducible datasets to facilitate the individualized prediction of immunotherapy response. Upskilling the pathology workforce through professional education and the recognition of computational pathology by professional societies will be essential to meet future clinical demands for optimal patient care.
VHK, KS, JR, and KDM jointly conceptualized this work, drafted the manuscript, and approved the final version to be published.
Viktor H Koelzer is supported by a research fellowship of the Swiss National Science Foundation (P2SKP3_168322). Kirsten D Mertz is supported by a grant of the Krebsliga beider Basel (KLbB-4182-03-2017) and by a grant of the “Freiwillige Akademische Gesellschaft Basel.” KS and JR were supported by the Oxford NIHR Oxford Biomedical Research Centre.
Compliance with ethical standards
No research involving human participants and/or animals was performed for this study.
Conflict of interest
KS and JR have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, royalties, or patents received or pending. V.H.K. and K.D.M. are participants of a patent application co-owned by the Netherlands Cancer Institute (NKI-AVL) and the University of Basel on the assessment of cancer immunotherapy biomarkers by digital pathology. Based on NKI-AVL and the University of Basel policy on management of intellectual property, V.H.K. and K.D.M. would be entitled to a portion of received royalty income. V.H.K. has served as an invited speaker on behalf of Indica Labs.
- 1.Directive 98/79/EC of the European Parliament and of the Council of 27 October 1998 on in vitro diagnostic medical devices. https://ec.europa.eu/growth/single-market/european-standards/harmonised-standards/iv-diagnostic-medical-devices_en. Accessed March 6th, 2018
- 2.Roche receives FDA clearance for the VENTANA MMR IHC panel for patients diagnosed with colorectal cancer. http://www.ventana.com/roche-receives-fda-clearance-ventana-mmr-ihc-panel-patients-diagnosed-colorectal-cancer/. Accessed May 14th, 2018
- 3.U.S. Department of Health and Human Services, Food and Drug Administration, Center for Devices and Radiological Health, Office of In Vitro Diagnostics and Radiological Health, Division of Molecular Genetics and Pathology MP and CB. [Internet]. Technical performance assessment of digital pathology whole slide imagingdevices; 2016. Available from: http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM435355.pdf. Accessed March 7th, 2018
- 4.Alexandrov LB, Nik-Zainal S, Wedge DC, Aparicio SA, Behjati S, Biankin AV, Bignell GR, Bolli N, Borg A, Borresen-Dale AL, Boyault S, Burkhardt B, Butler AP, Caldas C, Davies HR, Desmedt C, Eils R, Eyfjord JE, Foekens JA, Greaves M, Hosoda F, Hutter B, Ilicic T, Imbeaud S, Imielinski M, Jager N, Jones DT, Jones D, Knappskog S, Kool M, Lakhani SR, Lopez-Otin C, Martin S, Munshi NC, Nakamura H, Northcott PA, Pajic M, Papaemmanuil E, Paradiso A, Pearson JV, Puente XS, Raine K, Ramakrishna M, Richardson AL, Richter J, Rosenstiel P, Schlesner M, Schumacher TN, Span PN, Teague JW, Totoki Y, Tutt AN, Valdes-Mas R, van Buuren MM, van ‘t Veer L, Vincent-Salomon A, Waddell N, Yates LR, Australian Pancreatic Cancer Genome I, Consortium IBC, Consortium IM-S, PedBrain I, Zucman-Rossi J, Futreal PA, McDermott U, Lichter P, Meyerson M, Grimmond SM, Siebert R, Campo E, Shibata T, Pfister SM, Campbell PJ, Stratton MR (2013) Signatures of mutational processes in human cancer. Nature 500:415–421. https://doi.org/10.1038/nature12477 CrossRefGoogle Scholar
- 6.Bankhead P, Loughrey MB, Fernandez JA, Dombrowski Y, McArt DG, Dunne PD, McQuaid S, Gray RT, Murray LJ, Coleman HG, James JA, Salto-Tellez M, Hamilton PW (2017) QuPath: open source software for digital pathology image analysis. Sci Rep 7:16878. https://doi.org/10.1038/s41598-017-17204-5 CrossRefGoogle Scholar
- 8.Barua S, Fang P, Sharma A, Fujimoto J, Wistuba I, Rao AUK, Lin SH (2018) Spatial interaction of tumor cells and regulatory T cells correlates with survival in non-small cell lung cancer. Lung Cancer. https://doi.org/10.1016/j.lungcan.2018.01.022
- 12.Borghaei H, Paz-Ares L, Horn L, Spigel DR, Steins M, Ready NE, Chow LQ, Vokes EE, Felip E, Holgado E, Barlesi F, Kohlhaufl M, Arrieta O, Burgio MA, Fayette J, Lena H, Poddubskaya E, Gerber DE, Gettinger SN, Rudin CM, Rizvi N, Crino L, Blumenschein GR Jr, Antonia SJ, Dorange C, Harbison CT, Graf Finckenstein F, Brahmer JR (2015) Nivolumab versus docetaxel in advanced nonsquamous non-small-cell lung cancer. N Engl J Med 373:1627–1639. https://doi.org/10.1056/NEJMoa1507643 CrossRefGoogle Scholar
- 13.Brahmer J, Reckamp KL, Baas P, Crino L, Eberhardt WE, Poddubskaya E, Antonia S, Pluzanski A, Vokes EE, Holgado E, Waterhouse D, Ready N, Gainor J, Aren Frontera O, Havel L, Steins M, Garassino MC, Aerts JG, Domine M, Paz-Ares L, Reck M, Baudelet C, Harbison CT, Lestini B, Spigel DR (2015) Nivolumab versus docetaxel in advanced squamous-cell non-small-cell lung cancer. N Engl J Med 373:123–135. https://doi.org/10.1056/NEJMoa1504627 CrossRefGoogle Scholar
- 15.Capper D, Jones DTW, Sill M, Hovestadt V, Schrimpf D, Sturm D, Koelsche C, Sahm F, Chavez L, Reuss DE, Kratz A, Wefers AK, Huang K, Pajtler KW, Schweizer L, Stichel D, Olar A, Engel NW, Lindenberg K, Harter PN, Braczynski AK, Plate KH, Dohmen H, Garvalov BK, Coras R, Holsken A, Hewer E, Bewerunge-Hudler M, Schick M, Fischer R, Beschorner R, Schittenhelm J, Staszewski O, Wani K, Varlet P, Pages M, Temming P, Lohmann D, Selt F, Witt H, Milde T, Witt O, Aronica E, Giangaspero F, Rushing E, Scheurlen W, Geisenberger C, Rodriguez FJ, Becker A, Preusser M, Haberler C, Bjerkvig R, Cryan J, Farrell M, Deckert M, Hench J, Frank S, Serrano J, Kannan K, Tsirigos A, Bruck W, Hofer S, Brehmer S, Seiz-Rosenhagen M, Hanggi D, Hans V, Rozsnoki S, Hansford JR, Kohlhof P, Kristensen BW, Lechner M, Lopes B, Mawrin C, Ketter R, Kulozik A, Khatib Z, Heppner F, Koch A, Jouvet A, Keohane C, Muhleisen H, Mueller W, Pohl U, Prinz M, Benner A, Zapatka M, Gottardo NG, Driever PH, Kramm CM, Muller HL, Rutkowski S, von Hoff K, Fruhwald MC, Gnekow A, Fleischhack G, Tippelt S, Calaminus G, Monoranu CM, Perry A, Jones C, Jacques TS, Radlwimmer B, Gessi M, Pietsch T, Schramm J, Schackert G, Westphal M, Reifenberger G, Wesseling P, Weller M, Collins VP, Blumcke I, Bendszus M, Debus J, Huang A, Jabado N, Northcott PA, Paulus W, Gajjar A, Robinson GW, Taylor MD, Jaunmuktane Z, Ryzhova M, Platten M, Unterberg A, Wick W, Karajannis MA, Mittelbronn M, Acker T, Hartmann C, Aldape K, Schuller U, Buslei R, Lichter P, Kool M, Herold-Mende C, Ellison DW, Hasselblatt M, Snuderl M, Brandner S, Korshunov A, von Deimling A, Pfister SM (2018) DNA methylation-based classification of central nervous system tumours. Nature 555:469–474. https://doi.org/10.1038/nature26000 CrossRefGoogle Scholar
- 16.Conde E, Caminoa A, Dominguez C, Calles A, Walter S, Angulo B, Sanchez E, Alonso M, Jimenez L, Madrigal L, Hernando F, Sanz-Ortega J, Jimenez B, Garrido P, Paz-Ares L, de Castro J, Hernandez S, Lopez-Rios F (2018) Aligning digital CD8(+) scoring and targeted next-generation sequencing with programmed death ligand 1 expression: a pragmatic approach in early-stage squamous cell lung carcinoma. Histopathology 72:270–284. https://doi.org/10.1111/his.13346 CrossRefGoogle Scholar
- 17.Cooper WA, Russell PA, Cherian M, Duhig EE, Godbolt D, Jessup PJ, Khoo C, Leslie C, Mahar A, Moffat DF, Sivasubramaniam V, Faure C, Reznichenko A, Grattan A, Fox SB (2017) Intra- and Interobserver reproducibility assessment of PD-L1 biomarker in non-small cell lung cancer. Clin Cancer Res 23:4569–4577. https://doi.org/10.1158/1078-0432.CCR-17-0151 CrossRefGoogle Scholar
- 18.Cuzick J, Dowsett M, Pineda S, Wale C, Salter J, Quinn E, Zabaglo L, Mallon E, Green AR, Ellis IO, Howell A, Buzdar AU, Forbes JF (2011) Prognostic value of a combined estrogen receptor, progesterone receptor, Ki-67, and human epidermal growth factor receptor 2 immunohistochemical score and comparison with the Genomic Health recurrence score in early breast cancer. J Clin Oncol 29:4273–4278. https://doi.org/10.1200/JCO.2010.31.2835 CrossRefGoogle Scholar
- 19.Diem S, Hasan Ali O, Ackermann CJ, Bomze D, Koelzer VH, Jochum W, Speiser DE, Mertz KD, Flatz L (2018) Tumor infiltrating lymphocytes in lymph node metastases of stage III melanoma correspond to response and survival in nine patients treated with ipilimumab at the time of stage IV disease. Cancer Immunol Immunother 67:39–45. https://doi.org/10.1007/s00262-017-2061-4 CrossRefGoogle Scholar
- 20.Djuric U, Zadeh G, Aldape K, Diamandis P (2017) Precision histology: how deep learning is poised to revitalize histomorphology for personalized cancer care. NPJ Precis. Oncol. https://doi.org/10.1038/s41698-017-0022-1
- 21.Duruisseaux M, Martinez-Cardus A, Calleja-Cervantes ME, Moran S, Castro de Moura M, Davalos V, Pineyro D, Sanchez-Cespedes M, Girard N, Brevet M, Giroux-Leprieur E, Dumenil C, Pradotto M, Bironzo P, Capelletto E, Novello S, Cortot A, Copin MC, Karachaliou N, Gonzalez-Cao M, Peralta S, Montuenga LM, Gil-Bazo I, Baraibar I, Lozano MD, Varela M, Ruffinelli JC, Palmero R, Nadal E, Moran T, Perez L, Ramos I, Xiao Q, Fernandez AF, Fraga MF, Gut M, Gut I, Teixido C, Vilarino N, Prat A, Reguart N, Benito A, Garrido P, Barragan I, Emile JF, Rosell R, Brambilla E, Esteller M (2018) Epigenetic prediction of response to anti-PD-1 treatment in non-small-cell lung cancer: a multicentre, retrospective analysis. Lancet Respir Med 6:771–781. https://doi.org/10.1016/S2213-2600(18)30284-4 CrossRefGoogle Scholar
- 22.Ehteshami Bejnordi B, Veta M, Johannes van Diest P, van Ginneken B, Karssemeijer N, Litjens G, van der Laak J, the CC, Hermsen M, Manson QF, Balkenhol M, Geessink O, Stathonikos N, van Dijk MC, Bult P, Beca F, Beck AH, Wang D, Khosla A, Gargeya R, Irshad H, Zhong A, Dou Q, Li Q, Chen H, Lin HJ, Heng PA, Hass C, Bruni E, Wong Q, Halici U, Oner MU, Cetin-Atalay R, Berseth M, Khvatkov V, Vylegzhanin A, Kraus O, Shaban M, Rajpoot N, Awan R, Sirinukunwattana K, Qaiser T, Tsang YW, Tellez D, Annuscheit J, Hufnagl P, Valkonen M, Kartasalo K, Latonen L, Ruusuvuori P, Liimatainen K, Albarqouni S, Mungal B, George A, Demirci S, Navab N, Watanabe S, Seno S, Takenaka Y, Matsuda H, Ahmady Phoulady H, Kovalev V, Kalinovsky A, Liauchuk V, Bueno G, Fernandez-Carrobles MM, Serrano I, Deniz O, Racoceanu D, Venancio R (2017) Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer JAMA 318:2199–2210. doi: https://doi.org/10.1001/jama.2017.14585
- 25.Feng Z, Bethmann D, Kappler M, Ballesteros-Merino C, Eckert A, Bell RB, Cheng A, Bui T, Leidner R, Urba WJ, Johnson K, Hoyt C, Bifulco CB, Bukur J, Wickenhauser C, Seliger B, Fox BA (2017) Multiparametric immune profiling in HPV-oral squamous cell cancer. JCI Insight 2. https://doi.org/10.1172/jci.insight.93652
- 27.Fox BA, Schendel DJ, Butterfield LH, Aamdal S, Allison JP, Ascierto PA, Atkins MB, Bartunkova J, Bergmann L, Berinstein N, Bonorino CC, Borden E, Bramson JL, Britten CM, Cao X, Carson WE, Chang AE, Characiejus D, Choudhury AR, Coukos G, de Gruijl T, Dillman RO, Dolstra H, Dranoff G, Durrant LG, Finke JH, Galon J, Gollob JA, Gouttefangeas C, Grizzi F, Guida M, Hakansson L, Hege K, Herberman RB, Hodi FS, Hoos A, Huber C, Hwu P, Imai K, Jaffee EM, Janetzki S, June CH, Kalinski P, Kaufman HL, Kawakami K, Kawakami Y, Keilholtz U, Khleif SN, Kiessling R, Kotlan B, Kroemer G, Lapointe R, Levitsky HI, Lotze MT, Maccalli C, Maio M, Marschner JP, Mastrangelo MJ, Masucci G, Melero I, Melief C, Murphy WJ, Nelson B, Nicolini A, Nishimura MI, Odunsi K, Ohashi PS, O'Donnell-Tormey J, Old LJ, Ottensmeier C, Papamichail M, Parmiani G, Pawelec G, Proietti E, Qin S, Rees R, Ribas A, Ridolfi R, Ritter G, Rivoltini L, Romero PJ, Salem ML, Scheper RJ, Seliger B, Sharma P, Shiku H, Singh-Jasuja H, Song W, Straten PT, Tahara H, Tian Z, van Der Burg SH, von Hoegen P, Wang E, Welters MJ, Winter H, Withington T, Wolchok JD, Xiao W, Zitvogel L, Zwierzina H, Marincola FM, Gajewski TF, Wigginton JM, Disis ML (2011) Defining the critical hurdles in cancer immunotherapy. J Transl Med 9:214. https://doi.org/10.1186/1479-5876-9-214 CrossRefGoogle Scholar
- 29.Gerdes MJ, Sevinsky CJ, Sood A, Adak S, Bello MO, Bordwell A, Can A, Corwin A, Dinn S, Filkins RJ, Hollman D, Kamath V, Kaanumalle S, Kenny K, Larsen M, Lazare M, Li Q, Lowes C, McCulloch CC, McDonough E, Montalto MC, Pang Z, Rittscher J, Santamaria-Pang A, Sarachan BD, Seel ML, Seppo A, Shaikh K, Sui Y, Zhang J, Ginty F (2013) Highly multiplexed single-cell analysis of formalin-fixed, paraffin-embedded cancer tissue. Proc Natl Acad Sci USA 110:11982–11987. https://doi.org/10.1073/pnas.1300136110 CrossRefGoogle Scholar
- 30.Goltsev Y, Samusik N, Kennedy-Darling J, Bhate S, Hale M, Vasquez G, Black S, Nolan G (2018) Deep profiling of mouse splenic architecture with CODEX multiplexed imaging. bioRxiv. https://doi.org/10.1101/203166
- 32.Heindl A, Sestak I, Naidoo K, Cuzick J, Dowsett M, Yuan Y (2018) Relevance of spatial heterogeneity of immune infiltration for predicting risk of recurrence after endocrine therapy of ER+ breast cancer. J Natl Cancer Inst 110. https://doi.org/10.1093/jnci/djx137
- 42.Koelzer VH, Canonica K, Dawson H, Sokol L, Karamitopoulou-Diamantis E, Lugli A, Zlobec I (2016) Phenotyping of tumor-associated macrophages in colorectal cancer: impact on single cell invasion (tumor budding) and clinicopathological outcome. Oncoimmunology 5:e1106677. https://doi.org/10.1080/2162402X.2015.1106677 CrossRefGoogle Scholar
- 43.Koelzer VH, Dawson H, Andersson E, Karamitopoulou E, Masucci GV, Lugli A, Zlobec I (2015) Active immunosurveillance in the tumor microenvironment of colorectal cancer is associated with low frequency tumor budding and improved outcome. Transl Res 166:207–217. https://doi.org/10.1016/j.trsl.2015.02.008 CrossRefGoogle Scholar
- 44.Koelzer VH, Gisler A, Hanhart JC, Griss J, Wagner SN, Willi N, Cathomas G, Sachs M, Kempf W, Thommen DS, Mertz KD (2018) Digital image analysis improves precision of programmed death ligand 1 (PD-L1) scoring in cutaneous melanoma. Histopathology. https://doi.org/10.1111/his.13528
- 46.Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Proces Syst:1097–1105Google Scholar
- 47.Laubli H, Koelzer VH, Matter MS, Herzig P, Dolder Schlienger B, Wiese MN, Lardinois D, Mertz KD, Zippelius A (2018) The T cell repertoire in tumors overlaps with pulmonary inflammatory lesions in patients treated with checkpoint inhibitors. Oncoimmunology 7:e1386362. https://doi.org/10.1080/2162402X.2017.1386362 CrossRefGoogle Scholar
- 48.Lawler M, Alsina D, Adams RA, Anderson AS, Brown G, Fearnhead NS, Fenwick SW, Halloran SP, Hochhauser D, Hull MA, Koelzer VH, McNair AGK, Monahan KJ, Nathke I, Norton C, Novelli MR, Steele RJC, Thomas AL, Wilde LM, Wilson RH, Tomlinson I, Bowel Cancer UKCRGiCCI (2018) Critical research gaps and recommendations to inform research prioritisation for more effective prevention and improved outcomes in colorectal cancer. Gut 67:179–193. https://doi.org/10.1136/gutjnl-2017-315333 CrossRefGoogle Scholar
- 49.Le DT, Durham JN, Smith KN, Wang H, Bartlett BR, Aulakh LK, Lu S, Kemberling H, Wilt C, Luber BS, Wong F, Azad NS, Rucki AA, Laheru D, Donehower R, Zaheer A, Fisher GA, Crocenzi TS, Lee JJ, Greten TF, Duffy AG, Ciombor KK, Eyring AD, Lam BH, Joe A, Kang SP, Holdhoff M, Danilova L, Cope L, Meyer C, Zhou S, Goldberg RM, Armstrong DK, Bever KM, Fader AN, Taube J, Housseau F, Spetzler D, Xiao N, Pardoll DM, Papadopoulos N, Kinzler KW, Eshleman JR, Vogelstein B, Anders RA, Diaz LA Jr (2017) Mismatch repair deficiency predicts response of solid tumors to PD-1 blockade. Science 357:409–413. https://doi.org/10.1126/science.aan6733 CrossRefGoogle Scholar
- 51.LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE:2278–2324Google Scholar
- 53.Luksza M, Riaz N, Makarov V, Balachandran VP, Hellmann MD, Solovyov A, Rizvi NA, Merghoub T, Levine AJ, Chan TA, Wolchok JD, Greenbaum BD (2017) A neoantigen fitness model predicts tumour response to checkpoint blockade immunotherapy. Nature 551:517–520. https://doi.org/10.1038/nature24473 CrossRefGoogle Scholar
- 54.Ma Z, Shiao SL, Yoshida EJ, Swartwood S, Huang F, Doche ME, Chung AP, Knudsen BS, Gertych A (2017) Data integration from pathology slides for quantitative imaging of multiple cell types within the tumor immune cell infiltrate. Diagn Pathol 12:69. https://doi.org/10.1186/s13000-017-0658-8 CrossRefGoogle Scholar
- 56.Mezheyeuski A, Bergsland CH, Backman M, Djureinovic D, Sjoblom T, Bruun J, Micke P (2017) Multispectral imaging for quantitative and compartment-specific immune infiltrates reveals distinct immune profiles that classify lung cancer patients. J Pathol. https://doi.org/10.1002/path.5026
- 57.Mlecnik B, Van den Eynde M, Bindea G, Church SE, Vasaturo A, Fredriksen T, Lafontaine L, Haicheur N, Marliot F, Debetancourt D, Pairet G, Jouret-Mourin A, Gigot JF, Hubert C, Danse E, Dragean C, Carrasco J, Humblet Y, Valge-Archer V, Berger A, Pages F, Machiels JP, Galon J (2018) Comprehensive Intrametastatic immune quantification and major impact of immunoscore on survival. J Natl Cancer Inst 110. https://doi.org/10.1093/jnci/djx123
- 58.Mobadersany P, Yousefi S, Amgad M, Gutman DA, Barnholtz-Sloan JS, Velazquez Vega JE, Brat DJ, Cooper LAD (2018) Predicting cancer outcomes from histology and genomics using convolutional networks. Proc Natl Acad Sci U S A 115:E2970–E2979. https://doi.org/10.1073/pnas.1717139115 CrossRefGoogle Scholar
- 59.Moran S, Martinez-Cardus A, Sayols S, Musulen E, Balana C, Estival-Gonzalez A, Moutinho C, Heyn H, Diaz-Lagares A, de Moura MC, Stella GM, Comoglio PM, Ruiz-Miro M, Matias-Guiu X, Pazo-Cid R, Anton A, Lopez-Lopez R, Soler G, Longo F, Guerra I, Fernandez S, Assenov Y, Plass C, Morales R, Carles J, Bowtell D, Mileshkin L, Sia D, Tothill R, Tabernero J, Llovet JM, Esteller M (2016) Epigenetic profiling to classify cancer of unknown primary: a multicentre, retrospective analysis. Lancet Oncol 17:1386–1395. https://doi.org/10.1016/S1470-2045(16)30297-2 CrossRefGoogle Scholar
- 60.Motzer RJ, Escudier B, McDermott DF, George S, Hammers HJ, Srinivas S, Tykodi SS, Sosman JA, Procopio G, Plimack ER, Castellano D, Choueiri TK, Gurney H, Donskov F, Bono P, Wagstaff J, Gauler TC, Ueda T, Tomita Y, Schutz FA, Kollmannsberger C, Larkin J, Ravaud A, Simon JS, Xu LA, Waxman IM, Sharma P, CheckMate I (2015) Nivolumab versus everolimus in advanced renal-cell carcinoma. N Engl J Med 373:1803–1813. https://doi.org/10.1056/NEJMoa1510665 CrossRefGoogle Scholar
- 62.Overman MJ, McDermott R, Leach JL, Lonardi S, Lenz HJ, Morse MA, Desai J, Hill A, Axelson M, Moss RA, Goldberg MV, Cao ZA, Ledeine JM, Maglinte GA, Kopetz S, Andre T (2017) Nivolumab in patients with metastatic DNA mismatch repair-deficient or microsatellite instability-high colorectal cancer (CheckMate 142): an open-label, multicentre, phase 2 study. Lancet Oncol 18:1182–1191. https://doi.org/10.1016/S1470-2045(17)30422-9 CrossRefGoogle Scholar
- 63.Pagès F, Mlecnik B, Marliot F, Bindea G, Ou F-S, Bifulco C, Lugli A, Zlobec I, Rau TT, Berger MD, Nagtegaal ID, Vink-Börger E, Hartmann A, Geppert C, Kolwelter J, Merkel S, Grützmann R, Van den Eynde M, Jouret-Mourin A, Kartheuser A, Léonard D, Remue C, Wang JY, Bavi P, Roehrl MHA, Ohashi PS, Nguyen LT, Han S, MacGregor HL, Hafezi-Bakhtiari S, Wouters BG, Masucci GV, Andersson EK, Zavadova E, Vocka M, Spacek J, Petruzelka L, Konopasek B, Dundr P, Skalova H, Nemejcova K, Botti G, Tatangelo F, Delrio P, Ciliberto G, Maio M, Laghi L, Grizzi F, Fredriksen T, Buttard B, Angelova M, Vasaturo A, Maby P, Church SE, Angell HK, Lafontaine L, Bruni D, El Sissy C, Haicheur N, Kirilovsky A, Berger A, Lagorce C, Meyers JP, Paustian C, Feng Z, Ballesteros-Merino C, Dijkstra J, van de Water C, van Lent-van Vliet S, Knijn N, Mușină A-M, Scripcariu D-V, Popivanova B, Xu M, Fujita T, Hazama S, Suzuki N, Nagano H, Okuno K, Torigoe T, Sato N, Furuhata T, Takemasa I, Itoh K, Patel PS, Vora HH, Shah B, Patel JB, Rajvik KN, Pandya SJ, Shukla SN, Wang Y, Zhang G, Kawakami Y, Marincola FM, Ascierto PA, Sargent DJ, Fox BA, Galon J International validation of the consensus Immunoscore for the classification of colon cancer: a prognostic and accuracy study Lancet. https://doi.org/10.1016/S0140-6736(18)30789-X
- 64.Paik S, Shak S, Tang G, Kim C, Baker J, Cronin M, Baehner FL, Walker MG, Watson D, Park T, Hiller W, Fisher ER, Wickerham DL, Bryant J, Wolmark N (2004) A multigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer. N Engl J Med 351:2817–2826. https://doi.org/10.1056/NEJMoa041588 CrossRefGoogle Scholar
- 65.Pantanowitz L, Sinard JH, Henricks WH, Fatheree LA, Carter AB, Contis L, Beckwith BA, Evans AJ, Lal A, Parwani AV, College of American Pathologists P, Laboratory Quality C (2013) Validating whole slide imaging for diagnostic purposes in pathology: guideline from the College of American Pathologists Pathology and Laboratory Quality Center. Arch Pathol Lab Med 137:1710–1722. https://doi.org/10.5858/arpa.2013-0093-CP CrossRefGoogle Scholar
- 67.Salto-Tellez M, Maxwell P, Hamilton P (2018) Artificial intelligence—the third revolution in pathology. Histopathology. https://doi.org/10.1111/his.13760
- 68.Saltz J, Gupta R, Hou L, Kurc T, Singh P, Nguyen V, Samaras D, Shroyer KR, Zhao T, Batiste R, Van Arnam J, Cancer Genome Atlas Research N, Shmulevich I, AUK R, Lazar AJ, Sharma A, Thorsson V (2018) Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images. Cell Rep 23:181–193 e187. https://doi.org/10.1016/j.celrep.2018.03.086 CrossRefGoogle Scholar
- 70.Schubert W, Bonnekoh B, Pommer AJ, Philipsen L, Bockelmann R, Malykh Y, Gollnick H, Friedenberger M, Bode M, Dress AW (2006) Analyzing proteome topology and function by automated multidimensional fluorescence microscopy. Nat Biotechnol 24:1270–1278. https://doi.org/10.1038/nbt1250 CrossRefGoogle Scholar
- 71.Schulz D, Zanotelli VRT, Fischer JR, Schapiro D, Engler S, Lun XK, Jackson HW, Bodenmiller B (2018) Simultaneous multiplexed imaging of mrna and proteins with subcellular resolution in breast cancer tissue samples by mass cytometry. Cell Syst 6:25–36 e25. https://doi.org/10.1016/j.cels.2017.12.001 CrossRefGoogle Scholar
- 72.Sirinukunwattana K, Ahmed Raza SE, Yee-Wah T, Snead DR, Cree IA, Rajpoot NM (2016) Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imaging 35:1196–1206. https://doi.org/10.1109/TMI.2016.2525803 CrossRefGoogle Scholar
- 73.Sood A, Miller AM, Brogi E, Sui Y, Armenia J, McDonough E, Santamaria-Pang A, Carlin S, Stamper A, Campos C, Pang Z, Li Q, Port E, Graeber TG, Schultz N, Ginty F, Larson SM, Mellinghoff IK (2016) Multiplexed immunofluorescence delineates proteomic cancer cell states associated with metabolism. JCI Insight 1. https://doi.org/10.1172/jci.insight.87030
- 74.Spranger S, Luke JJ, Bao R, Zha Y, Hernandez KM, Li Y, Gajewski AP, Andrade J, Gajewski TF (2016) Density of immunogenic antigens does not explain the presence or absence of the T cell-inflamed tumor microenvironment in melanoma. Proc Natl Acad Sci USA 113:E7759–E7768. https://doi.org/10.1073/pnas.1609376113 CrossRefGoogle Scholar
- 76.Thommen DS, Koelzer VH, Herzig P, Roller A, Trefny M, Dimeloe S, Kiialainen A, Hanhart J, Schill C, Hess C, Savic Prince S, Wiese M, Lardinois D, Ho PC, Klein C, Karanikas V, Mertz KD, Schumacher TN, Zippelius A (2018) A transcriptionally and functionally distinct PD-1(+) CD8(+) T cell pool with predictive potential in non-small-cell lung cancer treated with PD-1 blockade. Nat Med 24:994–1004. https://doi.org/10.1038/s41591-018-0057-z CrossRefGoogle Scholar
- 78.Tsujikawa T, Kumar S, Borkar RN, Azimi V, Thibault G, Chang YH, Balter A, Kawashima R, Choe G, Sauer D, El Rassi E, Clayburgh DR, Kulesz-Martin MF, Lutz ER, Zheng L, Jaffee EM, Leyshock P, Margolin AA, Mori M, Gray JW, Flint PW, Coussens LM (2017) Quantitative multiplex immunohistochemistry reveals myeloid-inflamed tumor-immune complexity associated with poor prognosis. Cell Rep 19:203–217. https://doi.org/10.1016/j.celrep.2017.03.037 CrossRefGoogle Scholar
- 79.Tumeh PC, Harview CL, Yearley JH, Shintaku IP, Taylor EJ, Robert L, Chmielowski B, Spasic M, Henry G, Ciobanu V, West AN, Carmona M, Kivork C, Seja E, Cherry G, Gutierrez AJ, Grogan TR, Mateus C, Tomasic G, Glaspy JA, Emerson RO, Robins H, Pierce RH, Elashoff DA, Robert C, Ribas A (2014) PD-1 blockade induces responses by inhibiting adaptive immune resistance. Nature 515:568–571. https://doi.org/10.1038/nature13954 CrossRefGoogle Scholar
- 81.Virchow R (1858) Die Cellularpathologie in ihrer Begründung auf physiologische und pathologische Gewebelehre. August Hirschwald, BerlinGoogle Scholar
- 86.Yousefi S, Amrollahi F, Amgad M, Dong C, Lewis JE, Song C, Gutman DA, Halani SH, Velazquez Vega JE, Brat DJ, Cooper LAD (2017) Predicting clinical outcomes from large scale cancer genomic profiles with deep survival models. Sci Rep 7:11707. https://doi.org/10.1038/s41598-017-11817-6 CrossRefGoogle Scholar
- 87.Yuan Y (2015) Modelling the spatial heterogeneity and molecular correlates of lymphocytic infiltration in triple-negative breast cancer. J R Soc Interface 12. https://doi.org/10.1098/rsif.2014.1153
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.