Radiomics: A Well-Intentioned Leap of Faith

  • Clifford S. ChoEmail author
Hepatobiliary Tumors

Once in a while, someone teaches you something that forces you to re-evaluate the integrity of your intellectual foundations. If proven correct, this moment of cognitive disquietude can eventually transform our understanding of the world for the better, but the initial instinct is typically more akin to fear and trembling.

Two decades ago, Kattan et al. from the Memorial Sloan-Kettering Cancer Center gave their first demonstration of the prognostic nomogram.1 Now a familiar paradigm, it is easy to forget how conceptually disruptive the nomogram felt at the time. The idea of agnostically gathering any clinical or pathological characteristic, with no regard for the plausibility of its biological relevance, and dispassionately analyzing its incremental impact on survival, felt like such a radical departure from the scientific method of specific hypothesis-driven inquiry that has been promulgated since the Enlightenment. Yet, there were the data, with their impressive concordance index of 0.77, and if hair color would have improved the ability to predict one’s likelihood of succumbing to sarcoma, it would have been incorporated into the nomogram.1

Last year, DeepMind Technologies published the initial results of a new machine learning algorithm called AlphaZero.2 They programmed AlphaZero to teach itself the game of chess, not by learning conventional strategy and theory but through iterative, trial-and-error experience. Not only did AlphaZero learn how to play, but 9 h after it began learning, it soundly defeated Stockfish 8, the previous artificial intelligence chess champion. Science fiction and media hype about artificial intelligence may dull the emotional impact of news such as this, but consider the perspective of experts. After watching AlphaZero play, professionals who had spent the better part of their lives learning the intricacies of chess theory observed that it played a game that was somewhat unrecognizable. Adopting strikingly implausible strategies such as sacrificing one’s queen for the sake of positional advantage, observers described AlphaZero’s chess strategy as ‘alien’, ‘from another dimension’, or ‘as if a superior species landed on earth and showed us how they played chess’.3,4 By granting mathematical algorithms the opportunity to explore the process of discovery from scratch, we now have the ability to view once-familiar things in entirely new (and occasionally incomprehensible) ways.

In this issue of Annals of Surgical Oncology, Liao and colleagues from Sichuan University apply this brave new methodology to a subject with which we have plenty of familiarity—resected hepatocellular carcinoma (HCC) tumors.5 Using radiographic characteristics and features that have become the core of the emerging field of radiomics, they used machine learning algorithms to test whether imaging findings could predict the individual immunological landscape of HCC. Radiomics uses objective elements that can be deduced by radiographic imaging. Some of these elements (e.g. tumor size, shape, or brightness) can be approximated by the human observer. However, the majority of these elements (e.g. the textural connectedness between each individual pixel’s intensity and the one next to it, the randomness or non-randomness with which repetitive patterns of intratumoral heterogeneity duplicate themselves) cannot. By correlating imaging findings with post-resection immunohistochemistry and survival, the authors identifed a radiomic signature that correlates with anti-tumor immune responsiveness, as measured by CD8 + T cell infiltration and survival. Along with a handful of landmark papers published over the past year, this paper makes a convincing (albeit preliminary) argument that imaging can be used as a ‘digital biopsy’ to predict intratumoral features of prognostic and therapeutic significance.6 Sun and colleagues used a similar approach that took advantage of genomic data collected in a clinical trial to demonstrate how radiomics can establish a signature for solid tumors to predict not only CD8 + T-cell infiltration but also response to anti-PD-1 or anti-PD-L1 checkpoint inhibition immunotherapy.7 Trebeschi and co-authors made similar observations, and noted a correlation between morphological heterogeneity/non-uniform intratumoral density/compact borders and response to immunotherapy,8 leading to some tempting but ultimately blind speculation that high mutational burden and immunogenic inflammation (conditions that correlate with immunotherapy response) could be directly or indirectly detected by imaging. Chen and colleagues used magnetic resonance imaging to develop an HCC radiomic signature that, along with clinical variables such as tumor markers and transaminases, correlated with intratumoral CD3 + and CD8 + T-cell infiltration.9 A recent paper from the Memorial Sloan-Kettering Cancer Center similarly suggested that a combinatorial approach using clinical data plus radiomics could predict the presence of microsatellite instability in colorectal cancer.10

The clinical rationale for undertaking this investigative effort is self-evident. Checkpoint inhibition immunotherapy using monoclonal antibodies against CD8 + T-cell checkpoints such as CTLA-4 and PD-1 have transformed the prognosis and management of several advanced malignancies, and the viral etiology and relative immunogenicity of HCC have imbued a strong sense of hope for immunotherapy.11,12 Hurdles to broadening the role and impact of immunotherapy persist, and advances in radiomics will need to accompany advances in our understanding of cancer immunology. For example, Liao and co-authors found that their radiomic signature correlated with immune cell PD-1 expression, suggesting that these highly precise radiomic parameters were capable of identifying infiltrating CD8 + T cells, but incapable of distinguishing whether those infiltrating cells were immunologically effective or exhausted. Nevertheless, it would appear that conventionally available tools can already be used to identify patients expected to benefit from immunotherapy. Without too much creativity, one can envisage a scenario when radiomics will dictate rational treatment decision making. When that day arrives, the exact biological reason why a mathematical algorithm that dissects pixels on a two-dimensional surface is telling us to order a particular therapy may be beyond comprehension, or even relevance. In our never-ending pursuit to become better at relieving the pain and suffering of our fellow humans, our job will be to accept that new empiric reality—with a well-intentioned leap of faith.


  1. 1.

    It did not, and so it was not.



  1. 1.
    Kattan MW, Leung DH, Brennan MF. Postoperative nomogram for 12-year sarcoma-specific death. J Clin Oncol. 2002;20(3):791–796.CrossRefGoogle Scholar
  2. 2.
    Silver D, Hubert T, Schrittwieser J, et al. A general reinforcement learning algorithm that masters chess, shogi and Go through self-play. Science. 2018;362(6419);1140–1144.CrossRefGoogle Scholar
  3. 3.
    Knight W. “AlphaZero’s ‘alien’ chess shows the power, and the peculiarity, of AI”. MIT Technology Review. 8 Dec 2017;
  4. 4.
    “Google’s ‘superhuman’ DeepMind AI claims chess crown.” BBC News. 6 Dec 2017;
  5. 5.
    Liao H, Zhang Z, Chen J, et al. Preoperative radiomic approach to evaluate tumor infiltrating CD8 + T cells in HCC patients using contrast-enhanced CT. Ann Surg Oncol. 2019. Scholar
  6. 6.
    Banna GL, Olivier T, Rundo F, et al. The promise of digital biopsy for the prediction of tumor molecular features and clinical outcomes associated with immunotherapy. Front Med. 2019;6:172.CrossRefGoogle Scholar
  7. 7.
    Sun R, Limnkin EJ, Vakalopoulou M, et al. A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy; an imaging biomarker, retrospective multicohort study. Lancet Oncol. 2018;19(9):1180–1191.CrossRefGoogle Scholar
  8. 8.
    Trebeschi S, Drago SG, Birkbak NJ, et al. Predicting response to cancer immunotherapy using noninvasive radiomic biomarkers. Ann Oncol. 2019.
  9. 9.
    Chen S, Feng S, Wei J, et al. Pretreatment prediction of immunoscore in hepatocellular cancer: a radiomics-based clinical model based on Gd-EOB-DTPA-enhanced MRI imaging. Eur Radiol. 2019;29(8):4177–4187.CrossRefGoogle Scholar
  10. 10.
    Golia Pernicka JS, Gagniere J, Chakraborthy J, et al. Radiomics-based prediction of microsatellite instability in colorectal cancer at initial computed tomography evaluation. Abdom Radiol. 2019.
  11. 11.
    Obeid JM, Kung PR, Zayfudim VM, et al. Immunotherapy for hepatocellular carcinoma patients: is it ready for prime time? Cancer Immunol Immunother. 2018;67(2):161–174.CrossRefGoogle Scholar
  12. 12.
    Harding JJ. Immune checkpoint blockade in advanced hepatocellular carcinoma an update and clinical review of ongoing clinical trials. Future Oncol. 2018;14(22):2293–2302.CrossRefGoogle Scholar

Copyright information

© Society of Surgical Oncology 2019

Authors and Affiliations

  1. 1.Division of Hepatopancreatobiliary and Advanced Gastrointestinal SurgeryUniversity of Michigan Medical SchoolAnn ArborUSA
  2. 2.Surgical ServiceAnn Arbor VA HealthcareAnn ArborUSA

Personalised recommendations