External validation of a combined PET and MRI radiomics model for prediction of recurrence in cervical cancer patients treated with chemoradiotherapy

  • François LuciaEmail author
  • Dimitris Visvikis
  • Martin Vallières
  • Marie-Charlotte Desseroit
  • Omar Miranda
  • Philippe Robin
  • Pietro Andrea Bonaffini
  • Joanne Alfieri
  • Ingrid Masson
  • Augustin Mervoyer
  • Caroline Reinhold
  • Olivier Pradier
  • Mathieu Hatt
  • Ulrike Schick
Original Article



The aim of this study was to validate previously developed radiomics models relying on just two radiomics features from 18F-fluorodeoxyglucose positron emission tomography (PET) and magnetic resonance imaging (MRI) images for prediction of disease free survival (DFS) and locoregional control (LRC) in locally advanced cervical cancer (LACC).


Patients with LACC receiving chemoradiotherapy were enrolled in two French and one Canadian center. Pre-treatment imaging was performed for each patient. Multicentric harmonization of the two radiomics features was performed with the ComBat method. The models for DFS (using the feature from apparent diffusion coefficient (ADC) MRI) and LRC (adding one PET feature to the DFS model) were tuned using one of the French cohorts (n = 112) and applied to the other French (n = 50) and the Canadian (n = 28) external validation cohorts.


The DFS model reached an accuracy of 90% (95% CI [79–98%]) (sensitivity 92–93%, specificity 87–89%) in both the French and the Canadian cohorts. The LRC model reached an accuracy of 98% (95% CI [90–99%]) (sensitivity 86%, specificity 100%) in the French cohort and 96% (95% CI [80–99%]) (sensitivity 83%, specificity 100%) in the Canadian cohort. Accuracy was significantly lower without ComBat harmonization (82–85% and 71–86% for DFS and LRC, respectively). The best prediction using standard clinical variables was 56–60% only.


The previously developed PET/MRI radiomics predictive models were successfully validated in two independent external cohorts. A proposed flowchart for improved management of patients based on these models should now be confirmed in future larger prospective studies.


Radiomics Prediction Chemoradiotherapy Cervical cancer External validation 


Compliance with ethical standards

Conflict of interest

Authors François Lucia, Dimitris Visvikis, Martin Vallières, Marie-Charlotte Desseroit, Omar Miranda, Philippe Robin, Pietro Andrea Bonaffini, Joanne Alfieri, Ingrid Masson, Augustin Mervoyer, Caroline Reinhold, Olivier Pradier, Mathieu Hatt, Ulrike Schick declare that they have no conflict of interest.

No financial support was received for this work.

There are no potential conflicts of interest to disclose.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Statement of translational relevance describing how the results might be applied to the future practice of cancer medicine

Our findings have a direct impact on patient management in clinical practice. A flowchart demonstrates how to exploit the two radiomics features necessary to guide and personalize treatment: the textural feature (EntropyGLCM) extracted from ADC maps derived from DWI-MRI acquisitions can identify patients with low risk of recurrence, for which it could be advised to avoid adjuvant treatment. Among the patients with a higher risk of recurrence, the second textural feature (GLNUGLRLM) extracted from the FDG PET can differentiate between patients with a risk of distant recurrence, for which a systemic adjuvant treatment or more intensive surveillance could be recommended, and those with locoregional relapse, for which a locoregional adjuvant treatment might be more beneficial. Both MRI and PET images are routinely acquired for LACC patients, and the radiomics features have standardized definition with the IBSI guidelines, therefore anyone could easily evaluate our models in their data.

Supplementary material

259_2018_4231_MOESM1_ESM.docx (442 kb)
ESM 1 (DOCX 442 kb)


  1. 1.
    Ferlay J, Soerjomataram I, Dikshit R, Eser S, Mathers C, Rebelo M, et al. Cancer incidence and mortality worldwide: sources, methods and major patterns in GLOBOCAN 2012. Int J Cancer. 2015;136(5):E359–86.CrossRefGoogle Scholar
  2. 2.
    Moore KN, Java JJ, Slaughter KN, Rose PG, Lanciano R, DiSilvestro PA, et al. Is age a prognostic biomarker for survival among women with locally advanced cervical cancer treated with chemoradiation? An NRG oncology/gynecologic oncology group ancillary data analysis. Gynecol Oncol. 2016;143(2):294–301.CrossRefPubMedPubMedCentralGoogle Scholar
  3. 3.
    Herrera FG, Prior JO. The role of PET/CT in cervical cancer. Front Oncol. 2013;3:34.CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Choi J, Kim HJ, Jeong YH, Lee JH, Cho A, Yun M, et al. The role of (18) F-FDG PET/CT in assessing therapy response in cervix cancer after concurrent chemoradiation therapy. Nucl Med Mol Imaging. 2014;48(2):130–6.CrossRefPubMedGoogle Scholar
  5. 5.
    Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RG, Granton P, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012;48(4):441–6.CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Yip SS, Aerts HJ. Applications and limitations of radiomics. Phys Med Biol. 2016;61(13):R150–66.CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Ho KC, Fang YH, Chung HW, Yen TC, Ho TY, Chou HH, et al. A preliminary investigation into textural features of intratumoral metabolic heterogeneity in (18)F-FDG PET for overall survival prognosis in patients with bulky cervical cancer treated with definitive concurrent chemoradiotherapy. Am J Nucl Med Mol Imaging. 2016;6(3):166–75.PubMedPubMedCentralGoogle Scholar
  8. 8.
    Torheim T, Groendahl AR, Andersen EK, Lyng H, Malinen E, Kvaal K, et al. Cluster analysis of dynamic contrast enhanced MRI reveals tumor subregions related to locoregional relapse for cervical cancer patients. Acta Oncol. 2016;55(11):1294–8.CrossRefPubMedGoogle Scholar
  9. 9.
    Chung HH, Kang SY, Ha S, Kim JW, Park NH, Song YS, et al. Prognostic value of preoperative intratumoral FDG uptake heterogeneity in early stage uterine cervical cancer. J Gynecol Oncol. 2016;27(2):e15.CrossRefPubMedGoogle Scholar
  10. 10.
    Guan Y, Li W, Jiang Z, Chen Y, Liu S, He J, et al. Whole-lesion apparent diffusion coefficient-based entropy-related parameters for characterizing cervical cancers: initial findings. Acad Radiol. 2016;23(12):1559–67.CrossRefPubMedGoogle Scholar
  11. 11.
    Reuze S, Orlhac F, Chargari C, Nioche C, Limkin E, Riet F, et al. Prediction of cervical cancer recurrence using textural features extracted from 18F-FDG PET images acquired with different scanners. Oncotarget. 2017;8(26):43169–79.CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Lucia F, Visvikis D, Desseroit MC, Miranda O, Malhaire JP, Robin P, et al. Prediction of outcome using pretreatment (18)F-FDG PET/CT and MRI radiomics in locally advanced cervical cancer treated with chemoradiotherapy. Eur J Nucl Med Mol Imaging. 2018;45(5):768–86.CrossRefPubMedGoogle Scholar
  13. 13.
    Zwanenburg A, Lock S. Why validation of prognostic models matters? Radiother Oncol J Eur Soc Ther Radiol Oncol. 2018;127(3):370–3.CrossRefGoogle Scholar
  14. 14.
    Vallieres M, Zwanenburg A, Badic B, Cheze Le Rest C, Visvikis D, Hatt M. Responsible radiomics research for faster clinical translation. J Nucl Med. 2018;59(2):189–93.CrossRefPubMedPubMedCentralGoogle Scholar
  15. 15.
    Lim K, Small W Jr, Portelance L, Creutzberg C, Jurgenliemk-Schulz IM, Mundt A, et al. Consensus guidelines for delineation of clinical target volume for intensity-modulated pelvic radiotherapy for the definitive treatment of cervix cancer. Int J Radiat Oncol Biol Phys. 2011;79(2):348–55.CrossRefPubMedGoogle Scholar
  16. 16.
    Wahl RL, Jacene H, Kasamon Y, Lodge MA. From RECIST to PERCIST: evolving considerations for PET response criteria in solid tumors. J Nucl Med. 2009;50(Suppl 1):122S–50S.CrossRefPubMedPubMedCentralGoogle Scholar
  17. 17.
    Zwanenburg A, Leger S, Vallières M, Löck S. Image biomarker standardisation initiative - feature definitions. 2017.Google Scholar
  18. 18.
    Desseroit MC, et al. Comparison of three quantization methods for the calculation of textural features in PET/CT images: impact on prognostic models in non-small cell lung cancer. IEEE Nucl Sci Symp Med Imaging Conf 2016. 2016.Google Scholar
  19. 19.
    Chalkidou A, O'Doherty MJ, Marsden PK. False discovery rates in PET and CT studies with texture features: a systematic review. PLoS One. 2015;10(5):e0124165.CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Eur J Clin Investig. 2015;45(2):204–14.CrossRefGoogle Scholar
  21. 21.
    Yang F, Dogan N, Stoyanova R, Ford JC. Evaluation of radiomic texture feature error due to MRI acquisition and reconstruction: a simulation study utilizing ground truth. Phys Med Int J Devoted Appl Phys Med Biol Off J Ital Assoc Biomed Phys. 2018;50:26–36.Google Scholar
  22. 22.
    Yan J, Chu-Shern JL, Loi HY, Khor LK, Sinha AK, Quek ST, et al. Impact of image reconstruction settings on texture features in 18F-FDG PET. J Nucl Med. 2015;56(11):1667–73.CrossRefPubMedGoogle Scholar
  23. 23.
    Hatt M, Cheze le Rest C, Turzo A, Roux C, Visvikis D. A fuzzy locally adaptive Bayesian segmentation approach for volume determination in PET. IEEE Trans Med Imaging. 2009;28(6):881–93.CrossRefPubMedPubMedCentralGoogle Scholar
  24. 24.
    Hatt M, Cheze le Rest C, Descourt P, Dekker A, De Ruysscher D, Oellers M, et al. Accurate automatic delineation of heterogeneous functional volumes in positron emission tomography for oncology applications. Int J Radiat Oncol Biol Phys. 2010;77(1):301–8.CrossRefPubMedGoogle Scholar
  25. 25.
    Velazquez ER, Parmar C, Jermoumi M, Mak RH, van Baardwijk A, Fennessy FM, et al. Volumetric CT-based segmentation of NSCLC using 3D-slicer. Sci Rep. 2013;3:3529.CrossRefPubMedPubMedCentralGoogle Scholar
  26. 26.
    Fortin JP, Cullen N, Sheline YI, Taylor WD, Aselcioglu I, Cook PA, et al. Harmonization of cortical thickness measurements across scanners and sites. Neuroimage. 2018;167:104–20.CrossRefPubMedGoogle Scholar
  27. 27.
    Orlhac F, Boughdad S, Philippe C, Stalla-Bourdillon H, Nioche C, Champion L, Soussan M, Frouin F, Frouin V, Buvat I. A post-reconstruction harmonization method for multicenter radiomic studies in PET. J Nucl Med. 2018.Google Scholar
  28. 28.
    Johnson WE, Li C, Rabinovic A. Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics. 2007;8(1):118–27.CrossRefPubMedGoogle Scholar
  29. 29.
    Fortin JP, Parker D, Tunc B, Watanabe T, Elliott MA, Ruparel K, et al. Harmonization of multi-site diffusion tensor imaging data. NeuroImage. 2017;161:149–70.CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Naik A, Gurjar OP, Gupta KL, Singh K, Nag P, Bhandari V. Comparison of dosimetric parameters and acute toxicity of intensity-modulated and three-dimensional radiotherapy in patients with cervix carcinoma: a randomized prospective study. Cancer Radiother J de la Societe Francaise de Radiother Oncol. 2016;20(5):370–6.Google Scholar
  31. 31.
    Lin G, Yang LY, Lin YC, Huang YT, Liu FY, Wang CC, Lu HY, Chiang HJ, Chen YR, Wu RC, et al. Prognostic model based on magnetic resonance imaging, whole-tumour apparent diffusion coefficient values and HPV genotyping for stage IB-IV cervical cancer patients following chemoradiotherapy. Eur Radiol. 2018.Google Scholar
  32. 32.
    Galavis PE, Hollensen C, Jallow N, Paliwal B, Jeraj R. Variability of textural features in FDG PET images due to different acquisition modes and reconstruction parameters. Acta Oncol. 2010;49(7):1012–6.CrossRefPubMedPubMedCentralGoogle Scholar
  33. 33.
    Goh WWB, Wang W, Wong L. Why batch effects matter in omics data, and how to avoid them. Trends Biotechnol. 2017;35(6):498–507.CrossRefPubMedGoogle Scholar
  34. 34.
    Kothari S, Phan JH, Stokes TH, Osunkoya AO, Young AN, Wang MD. Removing batch effects from histopathological images for enhanced cancer diagnosis. IEEE J Biomed Health Inf. 2014;18(3):765–72.CrossRefGoogle Scholar
  35. 35.
    Mackin D, Fave X, Zhang L, Yang J, Jones AK, Ng CS, et al. Harmonizing the pixel size in retrospective computed tomography radiomics studies. PLoS One. 2017;12(9):e0178524.CrossRefPubMedPubMedCentralGoogle Scholar
  36. 36.
    Wu J, Aguilera T, Shultz D, Gudur M, Rubin DL, Loo BW Jr, et al. Early-stage non-small cell lung cancer: quantitative imaging characteristics of (18)F fluorodeoxyglucose PET/CT allow prediction of distant metastasis. Radiology. 2016;281(1):270–8.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • François Lucia
    • 1
    • 2
    Email author
  • Dimitris Visvikis
    • 2
  • Martin Vallières
    • 2
  • Marie-Charlotte Desseroit
    • 2
  • Omar Miranda
    • 1
  • Philippe Robin
    • 3
  • Pietro Andrea Bonaffini
    • 4
  • Joanne Alfieri
    • 5
  • Ingrid Masson
    • 6
  • Augustin Mervoyer
    • 6
  • Caroline Reinhold
    • 4
  • Olivier Pradier
    • 1
    • 2
  • Mathieu Hatt
    • 2
  • Ulrike Schick
    • 1
    • 2
  1. 1.Radiation Oncology DepartmentUniversity HospitalBrestFrance
  2. 2.LaTIM, INSERM, UMR 1101University BrestBrestFrance
  3. 3.Nuclear Medicine DepartmentUniversity HospitalBrestFrance
  4. 4.Department of RadiologyMcGill University Health Centre (MUHC)MontrealCanada
  5. 5.Department of Radiation OncologyMcGill University Health Centre (MUHC)MontrealCanada
  6. 6.Department of Radiation OncologyInstitut de Cancérologie de l’OuestNantesFrance

Personalised recommendations