European Radiology

, Volume 29, Issue 11, pp 6100–6108 | Cite as

Comparison of prediction models with radiological semantic features and radiomics in lung cancer diagnosis of the pulmonary nodules: a case-control study

  • Wei Wu
  • Larry A. Pierce
  • Yuzheng Zhang
  • Sudhakar N. J. Pipavath
  • Timothy W. Randolph
  • Kristin J. Lastwika
  • Paul D. Lampe
  • A. McGarry Houghton
  • Haining Liu
  • Liming XiaEmail author
  • Paul E. KinahanEmail author



To compare the ability of radiological semantic and quantitative texture features in lung cancer diagnosis of pulmonary nodules.

Materials and methods

A total of N = 121 subjects with confirmed non-small-cell lung cancer were matched with 117 controls based on age and gender. Radiological semantic and quantitative texture features were extracted from CT images with or without contrast enhancement. Three different models were compared using LASSO logistic regression: “CS” using clinical and semantic variables, “T” using texture features, and “CST” using clinical, semantic, and texture variables. For each model, we performed 100 trials of fivefold cross-validation and the average receiver operating curve was accessed. The AUC of the cross-validation study (AUCCV) was calculated together with its 95% confidence interval.


The AUCCV (and 95% confidence interval) for models T, CS, and CST was 0.85 (0.71–0.96), 0.88 (0.77–0.96), and 0.88 (0.77–0.97), respectively. After separating the data into two groups with or without contrast enhancement, the AUC (without cross-validation) of the model T was 0.86 both for images with and without contrast enhancement, suggesting that contrast enhancement did not impact the utility of texture analysis.


The models with semantic and texture features provided cross-validated AUCs of 0.85–0.88 for classification of benign versus cancerous nodules, showing potential in aiding the management of patients.

Key Points

• Pretest probability of cancer can aid and direct the physician in the diagnosis and management of pulmonary nodules in a cost-effective way.

• Semantic features (qualitative features reported by radiologists to characterize lung lesions) and radiomic (e.g., texture) features can be extracted from CT images.

• Input of these variables into a model can generate a pretest likelihood of cancer to aid clinical decision and management of pulmonary nodules.


Lung cancer Tomography Radiomics Semantics Statistical models 



Confidence intervals


X-ray computed tomography


Non-small cell lung cancer


The bounding volume maximum length


Volume of interest



At the University of Washington Medical Center, we thank Steven R. Bowen, PhD, for helpful suggestions on the project, Nina A. Mayr, MD, and William T. Yuh for providing the access to MIM software and guidance on how to use the software.


This study has received funding by NIH grants U01CA148131, U01185097, U01186157, P30CA015704, and F32CA200265, as well as National Natural Science Foundation of China (No. 81471637).

Compliance with ethical standards


The scientific guarantor of this publication is Paul E. Kinahan.

Conflict of interest

Paul E. Kinahan received a research grant from GE Healthcare outside of this work, and is the cofounder of PET/X LLC.

All other coauthors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

Timothy W. Randolph and Yuzheng Zhang (two coauthors in our paper) have significant statistical expertise.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.


• retrospective

• case-control study/diagnostic or prognostic study

• multicenter study

Supplementary material

330_2019_6213_MOESM1_ESM.docx (1.3 mb)
ESM 1 (DOCX 170 kb)


  1. 1.
    MacMahon H, Naidich DP, Goo JM et al (2017) Guidelines for management of incidental pulmonary nodules detected on CT images: from the Fleischner Society 2017. Radiology 284:228–243CrossRefGoogle Scholar
  2. 2.
    Carter SM, Barratt A (2017) What is overdiagnosis and why should we take it seriously in cancer screening? Public Health Res Pract 27:e2731722CrossRefGoogle Scholar
  3. 3.
    Edey AJ, Hansell DM (2009) Incidentally detected small pulmonary nodules on CT. Clin Radiol 64:872–884CrossRefGoogle Scholar
  4. 4.
    Dziedzic R, Rzyman W (2014) Incidentally diagnosed pulmonary nodules: a diagnostic algorithm. Kardiochir Torakochirurgia Pol 11:397–403PubMedPubMedCentralGoogle Scholar
  5. 5.
    Gould MK, Ananth L, Barnett PG (2007) A clinical model to estimate the pretest probability of lung cancer in patients with solitary pulmonary nodules. Chest 131:383–388CrossRefGoogle Scholar
  6. 6.
    Swensen SJ, Silverstein MD, Ilstrup DM, Schleck CD, Edell ES (2008) The probability of malignancy in solitary pulmonary nodules. Arch Intern Med 157:849–855Google Scholar
  7. 7.
    McWilliams A, Tammemagi MC, Mayo JR et al (2013) Probability of cancer in pulmonary nodules detected on first screening CT. N Engl J Med 369:910–919CrossRefGoogle Scholar
  8. 8.
    Schultz EM, Sanders GD, Trotter PR et al (2008) Validation of two models to estimate the probability of malignancy in patients with solitary pulmonary nodules. Thorax 63:335–341CrossRefGoogle Scholar
  9. 9.
    Al-Ameri A, Malhotra P, Thygesen H et al (2015) Risk of malignancy in pulmonary nodules: a validation study of four prediction models. Lung Cancer 89:27–30CrossRefGoogle Scholar
  10. 10.
    Talwar A, Rahman NM, Kadir T, Pickup LC, Gleeson F (2017) A retrospective validation study of three models to estimate the probability of malignancy in patients with small pulmonary nodules from a tertiary oncology follow-up center. Clin Radiol 72:177.e1–177.e8CrossRefGoogle Scholar
  11. 11.
    Gillies RJ, Kinahan PE, Hricak H (2016) Radiomics: images are more than pictures, they are data. Radiology 278:563–577CrossRefGoogle Scholar
  12. 12.
    van Griethuysen JJM, Fedorov A, Parmar C et al (2017) Computational radiomics system to decode the radiographic phenotype. Cancer Res 77:e104–e107CrossRefGoogle Scholar
  13. 13.
    Huang Y, Liu Z, He L et al (2016) Radiomics signature: a potential biomarker for the prediction of disease-free survival in early-stage (I or II) non—small cell lung cancer. Radiology 281:947–957CrossRefGoogle Scholar
  14. 14.
    Coroller TP, Agrawal V, Huynh E et al (2017) Radiomic-based pathological response prediction from primary tumors and lymph nodes in NSCLC. J Thorac Oncol 12:467–476CrossRefGoogle Scholar
  15. 15.
    Coroller TP, Grossmann P, Hou Y et al (2015) CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma. Radiother Oncol 114:345–350CrossRefGoogle Scholar
  16. 16.
    Huang YQ, Liang CH, He L et al (2016) Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer. J Clin Oncol 34:2157–2164Google Scholar
  17. 17.
    Song SH, Park H, Lee G et al (2017) Imaging phenotyping using radiomics to predict micropapillary pattern within lung adenocarcinoma. J Thorac Oncol 12:624–632CrossRefGoogle Scholar
  18. 18.
    Wilson R, Devaraj A (2017) Radiomics of pulmonary nodules and lung cancer. Transl Lung Cancer Res 6:86–91CrossRefGoogle Scholar
  19. 19.
    Hawkins S, Wang H, Liu Y et al (2016) Predicting malignant nodules from screening CT scans. J Thorac Oncol 11:2120–2128CrossRefGoogle Scholar
  20. 20.
    Chen CH, Chang CK, Tu CY et al (2018) Radiomic features analysis in computed tomography images of lung nodule classification. PLoS One 13:e0192002CrossRefGoogle Scholar
  21. 21.
    Hatt M, Tixier F, Pierce L, Kinahan PE, Le Rest CC, Visvikis D (2016) Characterization of PET/CT images using texture analysis: the past, the present… any future? Eur J Nucl Med Mol Imaging 44:151–165CrossRefGoogle Scholar
  22. 22.
    Soh LK, Tsatsoulis C (1999) Texture analysis of SAR sea ice imagery using gray level co- occurrence matrices. IEEE Trans Geosci Remote Sens 37:780–795CrossRefGoogle Scholar
  23. 23.
    Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Series B Stat Methodol 58:267–288Google Scholar
  24. 24.
    Friedman J, Hastie T, Tibshirani R (2010) Regularization paths for generalized linear models via coordinate descent. J Stat Softw 33:1–22CrossRefGoogle Scholar
  25. 25.
    Liu Y, Wang H, Li Q et al (2018) Radiologic features of small pulmonary nodules and lung cancer risk in the National Lung Screening Trial: a nested case-control study. Radiology 286:298–306CrossRefGoogle Scholar
  26. 26.
    Armato SG 3rd, McNitt-Gray MF, Reeves AP et al (2007) The Lung Image Database Consortium (LIDC): an evaluation of radiologist variability in the identification of lung nodules on CT scans. Acad Radiol 14:1409–1421Google Scholar
  27. 27.
    Kalpathy-Cramer J, Zhao B, Goldgof D et al (2016) A comparison of lung nodule segmentation algorithms: methods and results from a multi-institutional study. J Digit Imaging 29:476–487CrossRefGoogle Scholar
  28. 28.
    Herder GJ, van Tinteren H, Golding RP et al (2005) Clinical prediction model to characterize pulmonary nodules: validation and added value of 18F-fluorodeoxyglucose positron emission tomography. Chest 128:2490–2496CrossRefGoogle Scholar
  29. 29.
    He L, Huang Y, Ma Z, Liang C, Liang C, Liu Z (2016) Effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the diagnostic performance of radiomics signature in solitary pulmonary nodule. Sci Rep 6:34921CrossRefGoogle Scholar
  30. 30.
    Fan L, Fang M, Li Z et al (2019) Radiomics signature: a biomarker for the preoperative discrimination of lung invasive adenocarcinoma manifesting as a ground-glass nodule. Eur Radiol 29:889–897CrossRefGoogle Scholar
  31. 31.
    Larue RT, Defraene G, De Ruysscher D, Lambin P, Van Elmpt W (2017) Quantitative radiomics studies for tissue characterization: a review of technology and methodological procedures. Br J Radiol 90:20160665CrossRefGoogle Scholar
  32. 32.
    Zhao B, Tan Y, Tsai WY, Schwartz LH, Lu L (2014) Exploring variability in CT characterization of tumors: a preliminary phantom study. Transl Oncol 7:88–93CrossRefGoogle Scholar
  33. 33.
    Zhao B, Tan Y, Tsai WY et al (2016) Reproducibility of radiomics for deciphering tumor phenotype with imaging. Sci Rep 6:23428CrossRefGoogle Scholar
  34. 34.
    Lo P, Young S, Kim HJ, Brown MS, McNitt-Gray MF (2016) Variability in CT lung-nodule quantification: effects of dose reduction and reconstruction methods on density and texture based features. Med Phys 43:4854–4865CrossRefGoogle Scholar
  35. 35.
    Lu L, Ehmke RC, Schwartz LH, Zhao B (2016) Assessing agreement between radiomic features computed for multiple CT imaging settings. PLoS One 11:e0166550CrossRefGoogle Scholar
  36. 36.
    Shafiq-ul-Hassan M, Zhang GG, Latifi K et al (2017) Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels. Med Phys 44:1050–1062CrossRefGoogle Scholar
  37. 37.
    Mazzone PJ, Silvestri GA, Patel S et al (2018) Screening for lung cancer: CHEST guideline and expert panel report. Chest 153:954–985CrossRefGoogle Scholar
  38. 38.
    Hastie T, Tibshirani R, Sherlock G, Eisen M, Brown P, Botstein D (1999) Imputing missing data for gene expression arrays. Stanford University Statistics Department Technical report. URL: Last downloaded 2019-03-01

Copyright information

© European Society of Radiology 2019

Authors and Affiliations

  • Wei Wu
    • 1
    • 2
  • Larry A. Pierce
    • 1
  • Yuzheng Zhang
    • 3
  • Sudhakar N. J. Pipavath
    • 1
  • Timothy W. Randolph
    • 4
  • Kristin J. Lastwika
    • 5
    • 6
  • Paul D. Lampe
    • 5
    • 6
  • A. McGarry Houghton
    • 4
    • 6
    • 7
  • Haining Liu
    • 2
  • Liming Xia
    • 1
    Email author
  • Paul E. Kinahan
    • 2
    Email author
  1. 1.Department of Radiology, Tongji HospitalTongji Medical College affiliated to Huazhong University of Science and TechnologyWuhanPeople’s Republic of China
  2. 2.Department of RadiologyUniversity of WashingtonSeattleUSA
  3. 3.Program in Biostatistics and Biomathematics, Division of Public Health SciencesFred Hutchinson Cancer Research CenterSeattleUSA
  4. 4.Clinical Research DivisionFred Hutchinson Cancer Research CenterSeattleUSA
  5. 5.Translational Research Program, Public Health SciencesFred Hutchinson Cancer Research CenterSeattleUSA
  6. 6.Human Biology DivisionsFred Hutchinson Cancer Research CenterSeattleUSA
  7. 7.Division of Pulmonary and Critical CareUniversity of Washington Medical CenterSeattleUSA

Personalised recommendations