Skip to main content

Advertisement

SpringerLink
Log in

Why imaging data alone is not enough: AI-based integration of imaging, omics, and clinical data

  • Original Article
  • Published:
European Journal of Nuclear Medicine and Molecular Imaging Aims and scope Submit manuscript

Abstract

Artificial intelligence (AI) is currently regaining enormous interest due to the success of machine learning (ML), and in particular deep learning (DL). Image analysis, and thus radiomics, strongly benefits from this research. However, effectively and efficiently integrating diverse clinical, imaging, and molecular profile data is necessary to understand complex diseases, and to achieve accurate diagnosis in order to provide the best possible treatment. In addition to the need for sufficient computing resources, suitable algorithms, models, and data infrastructure, three important aspects are often neglected: (1) the need for multiple independent, sufficiently large and, above all, high-quality data sets; (2) the need for domain knowledge and ontologies; and (3) the requirement for multiple networks that provide relevant relationships among biological entities. While one will always get results out of high-dimensional data, all three aspects are essential to provide robust training and validation of ML models, to provide explainable hypotheses and results, and to achieve the necessary trust in AI and confidence for clinical applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

Notes

  1. https://www.regulations.gov/document?D=FDA-2019-N-1185-0001

References

  1. Marcus G. 2018. Deep learning: A critical appraisal. arXiv:1801.00631.

  2. Zhang Q, Yang LT, Chen Z, Li P. A survey on deep learning for big data. Inf Fus 2018;42:146–57.

    Google Scholar 

  3. Komaroff AL. The variability and inaccuracy of medical data. Proc IEEE 1979;67(9):1196–207.

    Google Scholar 

  4. Kadmon G, Pinchover M, Weissbach A, Kogan Hazan S, Nahum E. Case not closed: prescription errors 12 years after computerized physician order entry implementation. J Pediatr 2017;190 :236–40.e2. https://doi.org/10.1016/j.jpeds.2017.08.013.

    Article  PubMed  Google Scholar 

  5. Holzinger A, Jurisica I. Knowledge discovery and data mining in biomedical informatics: the future is in integrative, interactive machine learning solutions. Interactive knowledge discovery and data mining in biomedical informatics: state-of-the-art and future challenges. Lecture notes in computer science LNCS 8401. In: Holzinger A and Jurisica I, editors. Springer; 2014. p. 1–18.

  6. Wang B, et al. Similarity network fusion for aggregating data types on genomic scale. Nat Methods 2014;11 (3):3377.

    Google Scholar 

  7. Holzinger A. Biomedical informatics: discovering knowledge in big data. New York: Springer; 2014.

    Google Scholar 

  8. Hutson M. Artificial intelligence faces reproducibility crisis. Science 2018;359(6377):725–6.

    PubMed  Google Scholar 

  9. Goodman SN, Fanelli D, Ioannidis JPA. What does research reproducibility mean? Sci Transl Med 2016; 8(341):341ps12.

    PubMed  Google Scholar 

  10. Turkay C, Jeanquartier F, Holzinger A, Hauser H. On computationally-enhanced visual analysis of heterogeneous data and its application in biomedical informatics. Interactive knowledge discovery and data mining: state-of-the-art and future challenges in biomedical informatics. Lecture notes in computer science LNCS 8401. Springer; 2014. p. 117–140.

  11. Collins FS, Varmus H. A new initiative on precision medicine. England J Med 2015;372(9):793–5.

    CAS  Google Scholar 

  12. Schiphof D, Oei EH, Hofman A, Waarsing JH, Weinans H, Bierma-Zeinstra SM. Sensitivity and associations with pain and body weight of an MRI definition of knee osteoarthritis compared with radiographic Kellgren and Lawrence criteria: a population-based study in middle-aged females. Osteoarthritis Cartilage 2014;22(3): 440–6.

    CAS  PubMed  Google Scholar 

  13. Mlecnik B, Bindea G, Kirilovsky A, Angell HK, Obenauf AC, Tosolini M, Church SE, Maby P, Vasaturo A, Angelova M, Fredriksen T, Mauger S, Waldner M, Berger A, Speicher MR, Pages F, Valge-Archer V, Galon J. The tumor microenvironment and immunoscore are critical determinants of dissemination to distant metastasis. Sci Transl Med 2016;8(327):327ra26.

    PubMed  Google Scholar 

  14. Sanduleanu S, Woodruff HC, de Jong EEC, van Timmeren JE, Jochems A, Dubois L, Lambin P. Tracking tumor biology with radiomics: a systematic review utilizing a radiomics quality score. Radiotherapy and Oncology: Journal of the European Society for Therapeutic Radiology and Oncology 2018;127(3):349– 60.

    Google Scholar 

  15. Vargas HA, Veeraraghavan H, Micco M, Nougaret S, Lakhman Y, Meier AA, Sosa R, Soslow RA, Levine DA, Weigelt B, Aghajanian C, Hricak H, Deasy J, Snyder A, Sala E. A novel representation of inter-site tumour heterogeneity from pre-treatment computed tomography textures classifies ovarian cancers by clinical outcome. Eur Radiol 2017;27(9):3991– 4001.

    PubMed  PubMed Central  Google Scholar 

  16. Yin Q, Hung SC, Rathmell WK, Shen L, Wang L, Lin W, Fielding JR, Khandani AH, Woods ME, Milowsky MI, Brooks SA, Wallen EM, Shen D. Integrative radiomics expression predicts molecular subtypes of primary clear cell renal cell carcinoma. Clin Radiol 2018;73(9):782–91.

    CAS  PubMed  PubMed Central  Google Scholar 

  17. Liao Xin, Bo Cai, Tian Bin, Luo Yilin, Song Wen, Li Yinglong. 2019. Machine-learning based radiogenomics analysis of MRI features and metagenes in glioblastoma multiforme patients with different survival time. J Cell Mol Med, 4.

  18. Danala G, Thai T, Gunderson CC, Moxley KM, Moore K, Mannel RS, Liu H, Zheng B, Qiu Y. Applying quantitative CT image feature analysis to predict response of ovarian cancer patients to chemotherapy. Acad Radiol 2017;24(10):1233–9.

    PubMed  PubMed Central  Google Scholar 

  19. Cook GJR, Azad G, Owczarczyk K, Siddique M, Goh V. Challenges and promises of PET radiomics. Int J Radiat Oncol Biol Phys 2018;102(4):1083–89.

    PubMed  PubMed Central  Google Scholar 

  20. Lin G, Lai CH, Yen TC. Emerging molecular imaging techniques in gynecologic oncology. PET Clin 2018;13(2):289– 99.

    PubMed  Google Scholar 

  21. Xu Y, Hosny A, Zeleznik R, Parmar C, Coroller T, Franco I, Mak RH, Aerts HJWL. 2019. Deep learning predicts lung cancer treatment response from serial medical imaging. Clinical Cancer Research.

  22. Baessler B, Weiss K, Pinto Dos Santos D. Robustness and reproducibility of radiomics in magnetic resonance imaging: a phantom study. Invest Radiol 2019;54(4):221–8.

    PubMed  Google Scholar 

  23. Lau SK, Boutros PC, Pintilie M, Blackhall FH, Zhu CQ, Strumpf D, Johnston MR, Darling G, Keshavjee S, Waddell TK, Liu N, Lau D, Penn LZ, Shepherd FA, Jurisica I, Der SD, Tsao MS. Three-gene prognostic classifier for early-stage non small-cell lung cancer. J Clin Oncol 2007;25 (35):5562–9.

    PubMed  Google Scholar 

  24. Shedden K, Taylor JM, Enkemann SA, Tsao MS, Yeatman TJ, Gerald WL, Eschrich S, Jurisica I, Giordano TJ, Misek DE, Chang AC, Zhu CQ, Strumpf D, Hanash S, Shepherd FA, Ding K, Seymour L, Naoki K, Pennell N, Weir B, Verhaak R, Ladd-Acosta C, Golub T, Gruidl M, Sharma A, Szoke J, Zakowski M, Rusch V, Kris M, Viale A, Motoi N, Travis W, Conley B, Seshan VE, Meyerson M, Kuick R, Dobbin KK, Lively T, Jacobson JW, Beer DG. Gene expression-based survival prediction in lung adenocarcinoma: a multi-site, blinded validation study. Nat Med 2008;14(8):822–7.

    CAS  PubMed  PubMed Central  Google Scholar 

  25. Boutros PC, Lau SK, Pintilie M, Liu N, Shepherd FA, Der SD, Tsao MS, Penn LZ, Jurisica I. Prognostic gene signatures for non-small-cell lung cancer. Proc Natl Acad Sci USA 2009;106(8):2824–8.

    CAS  PubMed  PubMed Central  Google Scholar 

  26. Beaulieu-Jones BK, Greene CS. Reproducibility of computational workflows is automated using continuous analysis. Nat Biotechnol 2017;34(4):342–6.

    Google Scholar 

  27. Munafò MR, Nosek BA, Bishop VMD, Button SB, Chambers CD, du Sert N, Simonsohn U, Wagenmakers E-J, Ware JJ, Ioannidis JPA. A manifesto for reproducible science. Nat Hum Behav 2017; 1(1):s41562–016–0021.

    Google Scholar 

  28. Lamb J, Crawford ED, Peck D, Modell JW, Blat IC, Wrobel MJ, Lerner J, Brunet J-P, Subramanian A, Ross KN, Reich M, Hieronymus H, Wei G, Armstrong SA, Haggarty SJ, Clemons PA, Wei R, Carr SA, Lander ES, Golub TR. The connectivity map: using gene-expression signatures to connect small molecules, genes, and disease. Science 2006;313(5795):1929–35.

    CAS  PubMed  Google Scholar 

  29. Sun R, Limkin EJ, Vakalopoulou M, Dercle L, Champiat S, Han SR, Verlingue L, Brandao D, Lancia A, Ammari S, Hollebecque A, Scoazec JY, Marabelle A, Massard C, Soria JC, Robert C, Paragios N, Deutsch E, Ferte C. A radiomics approach to assess tumour-infiltrating cd8 cells and response to anti-pd-1 or anti-pd-l1 immunotherapy: an imaging biomarker, retrospective multicohort study. Lancet Oncol 2018;19(9):1180–91.

    CAS  PubMed  Google Scholar 

  30. Aerts HJ, Grossmann P, Tan Y, Oxnard GR, Rizvi N, Schwartz LH, Zhao B. Defining a radiomic response phenotype: a pilot study using targeted therapy in NSCLC. Sci Rep 2016;6:33860.

    CAS  PubMed  PubMed Central  Google Scholar 

  31. Tokar T, Pastrello C, Ramnarine VR, Zhu CQ, Craddock KJ, Pikor LA, Vucic EA, Vary S, Shepherd FA, Tsao MS, Lam WL, Jurisica I. Differentially expressed microRNAs in lung adenocarcinoma invert effects of copy number aberrations of prognostic genes. Oncotarget 2018;9(10):9137–55.

    PubMed  PubMed Central  Google Scholar 

  32. Viswanath SE, Tiwari P, Lee G, Madabhushi A. Dimensionality reduction-based fusion approaches for imaging and non-imaging biomedical data: concepts, workflow, and use-cases. BMC Med Imaging 2017;17(1):2.

    PubMed  PubMed Central  Google Scholar 

  33. Ingalhalikar M, Smith A, Parker D, Satterthwaite TD, Elliott MA, Ruparel K, Hakonarson H, Gur RE, Gur RC, Verma R. Sex differences in the structural connectome of the human brain. Proc Natl Acad Sci USA 2014;111(2):823–8.

    CAS  PubMed  Google Scholar 

  34. Bigler ED. Systems biology, neuroimaging, neuropsychology, neuroconnectivity and traumatic brain injury. Front Syst Neurosci 2016;10:55.

    PubMed  PubMed Central  Google Scholar 

  35. Cui LB, Liu L, Wang HN, Wang LX, Guo F, Xi YB, Liu TT, Li C, Tian P, Liu K, Wu WJ, Chen YH, Qin W, Yin H. Disease definition for schizophrenia by functional connectivity using radiomics strategy. Schizophrenia Bull 2018;44(5):1053–9.

    Google Scholar 

  36. Jeong H, Mason SP, Barabasi AL, Oltvai ZN. Lethality and centrality in protein networks. Nature 2001;411(6833):41–2.

    CAS  PubMed  Google Scholar 

  37. Przulj N, Wigle DA, Jurisica I. Functional topology in a network of protein interactions. Bioinformatics 2004;20(3):340–8.

    CAS  PubMed  Google Scholar 

  38. Ye P, Peyser BD, Pan X, Boeke JD, Spencer FA, Bader JS. Gene function prediction from congruent synthetic lethal interactions in yeast. Molec Syst Biol 2005;1:1.

    Google Scholar 

  39. Sharan R, Ulitsky I, Shamir R. 2007. Network-based prediction of protein function. Mol Syst Biol, 3.

    PubMed  PubMed Central  Google Scholar 

  40. Ulitsky I, Shamir R. 2007. Identification of functional modules using network topology and high-throughput data. BMC Syst Biol, 1.

  41. Chuang HY, Lee E, Liu YT, Lee D, Ideker T. 2007. Network-based classification of breast cancer metastasis. Mol Syst Biol, 3.

    PubMed  PubMed Central  Google Scholar 

  42. Fortney K, Kotlyar M, Jurisica I. Inferring the functions of longevity genes with modular subnetwork biomarkers of Caenorhabditis elegans aging. Genome Biol 2010;11(2):R13.

    PubMed  PubMed Central  Google Scholar 

  43. Kotlyar M, Fortney K, Jurisica I. Network-based characterization of drug-regulated genes, drug targets, and toxicity. Methods 2012;57(4):499–507. Protein-Protein Interactions.

    CAS  PubMed  Google Scholar 

  44. Hinze L, Pfirrmann M, Karim S, Degar J, McGuckin C, Vinjamur D, Sacher J, Stevenson KE, Neuberg DS, Orellana E, Stanulla M, Gregory RI, Bauer DE, Wagner FF, Stegmaier K, Gutierrez A. Synthetic lethality of Wnt pathway activation and asparaginase in drug-resistant acute leukemias. Cancer Cell 2019;35(4):664–676.e667.

    CAS  PubMed  PubMed Central  Google Scholar 

  45. Wong SWH, Pastrello C, Kotlyar M, Faloutsos C, Jurisica I. Sdregion: fast spotting of changing communities in biological networks. Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining. New York: ACM; 2018. p. 867–875.

  46. Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RG, Granton P, Zegers CM, Gillies R, Boellard R, Dekker A, Aerts HJ. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 2012;48(4):441–6.

    PubMed  PubMed Central  Google Scholar 

  47. Ger RB, Zhou S, Chi PM, Lee HJ, Layman RR, Jones AK, Goff DL, Fuller CD, Howell RM, Li H, Stafford RJ, Court LE, Mackin DS. Comprehensive investigation on controlling for CT imaging variabilities in radiomics studies. Sci Rep 2018;8(1):13047.

    PubMed  PubMed Central  Google Scholar 

  48. Jorgensen P, Nelson B, Robinson MD, Chen Y, Andrews B, Tyers M, Boone C. High-resolution genetic mapping with ordered arrays of Saccharomyces cerevisiae deletion mutants. Genetics 2002;162 (3):1091–9.

    CAS  PubMed  PubMed Central  Google Scholar 

  49. Suthers PF, Zomorrodi A, Maranas CD. 2009. Genome-scale gene/reaction essentiality and synthetic lethality analysis, Vol. 5.

    PubMed  PubMed Central  Google Scholar 

  50. Chan EM, Shibue T, McFarland JM, Gaeta B, Ghandi M, Dumont N, Gonzalez A, McPartlan JS, Li T, Zhang Y, Liu JB, Lazaro J-B, Gu P, Piett CG, Apffel A, Ali SO, Deasy R, Keskula P, Ng RWS, Roberts EA, Reznichenko E, Leung L, Alimova M, Schenone M, Islam M, Maruvka YE, Liu Y, Roper J, Raghavan S, Giannakis M, Tseng Y-Y, Nagel ZD, D’Andrea A, Root DE, Boehm JS, Getz G, Chang S, Golub TR, Tsherniak A, Vazquez F, Bass AJ. Wrn helicase is a synthetic lethal target in microsatellite unstable cancers. Nature 2019;568(7753): 551–6.

    CAS  PubMed  PubMed Central  Google Scholar 

  51. Holzinger A, Langs G, Denk H, Zatloukal K, Mueller H. 2019. Causability and explainability of AI in medicine. Wiley Interdisciplinary Reviews, Data Mining and Knowledge Discovery.

  52. Krittanawong C, Zhang HJ, Wang Z, Aydar M, Kitai T. Artificial intelligence in precision cardiovascular medicine. J Am Coll Cardiol 2017;69(21):2657–64.

    PubMed  Google Scholar 

  53. Hamet P, Tremblay J. Artificial intelligence in medicine. Metabolism 2017;69:S36–40.

    CAS  Google Scholar 

  54. Robertson S, Azizpour H, Smith K, Hartman J. Digital image analysis in breast pathology—from image processing techniques to artificial intelligence. Transl Res 2018;194:19–35.

    PubMed  Google Scholar 

  55. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med 2019;25 (1):44–56.

    CAS  PubMed  Google Scholar 

  56. Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts H. Artificial intelligence in radiology. Nat Rev Cancer 2018;18(8):500–510.

    CAS  PubMed  PubMed Central  Google Scholar 

  57. Kang J, Rancati T, Lee S, Oh J, Kerns SL, Scott JG, Schwartz R, Kim S, Rosenstein BS. Machine learning and radiogenomics: lessons learned and future directions. Frontiers Oncol 2018;8:228.

    Google Scholar 

  58. Sun R, Limkin E, Vakalopoulou M, Dercle L, Champiat S, Han S, Verlingue L, Brandao D, Lancia A, Ammari S, Hollebecque A, Scoazec J-Y, Marabelle A, Massard C, Soria J-C, Robert C, Paragios N, Deutsch E, Ferté C. A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: an imaging biomarker, retrospective multicohort study. Lancet Oncol 2018;19(9):1180–91.

    CAS  PubMed  Google Scholar 

  59. Iorio F, Knijnenburg TA, Vis DJ, Bignell GR, Menden MP, Schubert M, Aben N, Gonçalves E, Barthorpe S, Lightfoot H, Cokelaer T, Greninger P, van Dyk E, Chang H, de Silva H, Heyn H, Deng X, Egan RK, Liu Q, Mironenko T, Mitropoulos X, Richardson L, Wang J, Zhang T, Moran S, Sayols S, Soleimani M, Tamborero D, López-Bigas N, Ross-Macdonald P, Esteller M, Gray NS, Haber DA, Stratton MR, Benes CH, Wessels LFA, Saez-Rodriguez J, McDermott U, Garnett MJ. A landscape of pharmacogenomic interactions in cancer. Cell 2016;166(3):740–54.

    CAS  PubMed  PubMed Central  Google Scholar 

  60. Seashore-Ludlow B, Rees MG, Cheah JH, Cokol M, Price EV, Coletti ME, Jones V, Bodycombe NE, Soule CK, Gould J, Alexander B, Li A, Montgomery P, Wawer MJ, Kuru N, Kotz JD, Hon S-YC, Munoz B, Liefeld T, Dancik V, Bittker JA, Palmer M, Bradner JE, Shamji AF, Clemons PA, Schreiber SL. Harnessing connectivity in a large-scale small-molecule sensitivity dataset. Cancer Discovery.

  61. Haverty PM, Lin E, Tan J, Yu Y, Lam B, Lianoglou S, Neve RM, Martin S, Settleman J, Yauch RL, Bourgon R. Reproducible pharmacogenomic profiling of cancer cell line panels. Nature 2016; 533(7603):333–7.

    CAS  PubMed  Google Scholar 

  62. Gao H, Korn JM, Ferretti S, Monahan JE, Wang Y, Singh M, Zhang C, Schnell C, Yang G, Zhang Y, Balbin AO, Barbe S, Cai H, Casey F, Chatterjee S, Chiang DY, Chuai S, Cogan SM, Collins SD, Dammassa E, Ebel N, Embry M, Green J, Kauffmann A, Kowal C, Leary RJ, Lehar J, Liang Y, Loo Alice, Lorenzana E III, Robert E, McLaughlin ME, Merkin J, Meyer R, Naylor TL, Patawaran M, Reddy A, Röelli C, Ruddy DA, Salangsang F, Santacroce F, Singh AP, Tang Y, Tinetto W, Tobler S, Velazquez R, Venkatesan K, Arx F, Wang H, Wang Z, Wiesmann M, Wyss D, Xu F, Bitter H, Atadja P, Lees E, Hofmann F, Li E, Keen N, Cozens R, Jensen M, Pryer NK, Williams JA, Sellers WR. High-throughput screening using patient-derived tumor xenografts to predict clinical trial drug response. Nat Med 2015;21(11):1318–25.

    CAS  PubMed  Google Scholar 

  63. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521(7553):436.

    CAS  PubMed  Google Scholar 

  64. Chartrand G, Cheng PM, Vorontsov E, Drozdzal M, Turcotte S, Pal CJ, Kadoury S, An T. Deep learning A primer for radiologists. Radiographics Rev Publ Radiological Soc North Am Inc 2017;37(7): 2113–31.

    Google Scholar 

  65. Sharp G, Fritscher KD, Pekar V, Peroni M, Shusharina N, Veeraraghavan H, Yang J. Vision 20/20: perspectives on automated image segmentation for radiotherapy. Med Phys 2014;41(5):050902.

    PubMed  PubMed Central  Google Scholar 

  66. Cardenas CE, Yang J, Anderson BM, Court LE, Brock KB. Advances in auto-segmentation. Semin Radiat Oncol 2019;29(3):185–97. Adaptive Radiotherapy and Automation.

    PubMed  Google Scholar 

  67. Kreimeyer K, Foster M, Pandey A, Arya N, Halford G, Jones SF, Forshee R, Walderhaug M, Botsis T. 2017. Natural language processing systems for capturing and standardizing unstructured clinical information: a systematic review. J Biomed Inform, 73.

  68. Asim M, Wasim M, Khan M, Mahmood W, Abbasi H. 2018. A survey of ontology learning techniques and applications. Database, 2018.

  69. Deshpande PR, Rajan S, Sudeepthi LB, Abdul CN. Patient-reported outcomes: a new era in clinical research. Perspectives Clin Res 2011;2(4):137–44.

    Google Scholar 

  70. Loh E. Medicine and the rise of the robots: a qualitative review of recent advances of artificial intelligence in health. BMJ Leader 2018;2(2):59–63.

    Google Scholar 

  71. Dias D, Cunha J. Wearable health Devices—Vital sign monitoring, systems and technologies. Sensors 2018; 18(8):2414.

    PubMed Central  Google Scholar 

  72. Guyon I, Bennett K, Cawley G, Escalante HJ, Escalera S, Ho TK, Macia N, Ray B, Saeed M, Statnikov A. Design of the 2015 ChaLearn AutoML Challenge. 2015 International joint conference on neural networks (IJCNN). IEEE; 2015. p. 1–8.

  73. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D. Mastering the game of go with deep neural networks and tree search. Nature 2016;529(7587):484–9.

    CAS  PubMed  Google Scholar 

  74. Stickel C, Ebner M, Steinbach-Nordmann S, Searle G, Holzinger A. Emotion detection: application of the valence arousal space for rapid biological usability testing to enhance universal access. Universal access in human–computer interaction. Addressing diversity, lecture notes in computer science, LNCS 5614. In: Stephanidis C, editors. Springer; 2009. p. 615–24.

  75. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017;542(7639):115–8.

    CAS  PubMed  PubMed Central  Google Scholar 

  76. Holzinger A. Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Inform 2016;3(2):119–31.

    PubMed  PubMed Central  Google Scholar 

  77. Holzinger A, Stocker C, Ofner B, Prohaska G, Brabenetz A, Hofmann-Wellenhof R. Combining HCI, natural language processing, and knowledge discovery - potential of IBM content analytics as an assistive technology in the biomedical domain. Springer lecture notes in computer science LNCS 7947. Springer; 2013.

  78. Holzinger A, Plass M, Holzinger K, Crisan GC, Pintea C-M, Palade V. Towards interactive machine learning (IML): applying ant colony algorithms to solve the traveling salesman problem with the human-in-the-loop approach. Springer lecture notes in computer science LNCS 9817. Springer; 2016. p. 81–95.

Download references

Funding

This study was funded in part by Ontario Research Fund (34876) to IJ. B.H.K. was supported by the Gattuso-Slaight Personalized Cancer Medicine Fund at Princess Margaret Cancer Centre and the Artificial Intelligence and Microbiome Program supported by the Princess Margaret Cancer Foundation.

Author information

Authors and Affiliations

  1. Institute for Medical Informatics / Statistics, Medical University Graz, Auenbruggerplatz 2/V, 8036, Graz, Austria

    Andreas Holzinger

  2. Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada

    Benjamin Haibe-Kains

  3. Departments of Medical Biophysics and Computer Science, University of Toronto, Toronto, Canada

    Benjamin Haibe-Kains & Igor Jurisica

  4. Krembil Research Institute, UHN, 60 Leonard Avenue, 5KD-407, Toronto, Ontario, M5T 0S8, Canada

    Igor Jurisica

  5. Institute of Neuroimmunology, Slovak Academy of Sciences, Bratislava, Slovakia

    Igor Jurisica

Authors
  1. Andreas Holzinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benjamin Haibe-Kains
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Igor Jurisica
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Igor Jurisica.

Ethics declarations

Conflict of interest

Author AH declares no conflict of interest. Author BHK declares no conflict of interest. Author IJ declares no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Advanced Image Analyses (Radiomics and Artificial Intelligence)

Appendix: Abbreviations

Appendix: Abbreviations

AI ...:

Artificial Intelligence

CPU ...:

Central Processing Unit

CT ...:

Computer Tomography

ECG ...:

Electrocardiography

EEG ...:

Electroencephalography

EOG ...:

Electrooculography

EPR ...:

Electronic Patient Record

fMRI ...:

Functional Magnetic Resonance Imaging

GPU ...:

raphical Processing Unit

ICD ...:

International Classification of Diseases

ML ...:

Machine Learning

MRI ...:

Magnetic Resonance Imaging

NGS ...:

Next-Generation Sequencing

SNOMED CT ...:

Standard Nomenclature of Medicine Clinical Terms

TILs ...:

Tumor Infiltrated Lymphocytes

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Holzinger, A., Haibe-Kains, B. & Jurisica, I. Why imaging data alone is not enough: AI-based integration of imaging, omics, and clinical data. Eur J Nucl Med Mol Imaging 46, 2722–2730 (2019). https://doi.org/10.1007/s00259-019-04382-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00259-019-04382-9

Keywords

Search

Navigation