Skip to main content

Advertisement

SpringerLink
Log in

Integrating manual diagnosis into radiomics for reducing the false positive rate of 18F-FDG PET/CT diagnosis in patients with suspected lung cancer

  • Original Article
  • Published:
European Journal of Nuclear Medicine and Molecular Imaging Aims and scope Submit manuscript

Abstract

Purpose

The high false positive rate (FPR) of 18F-FDG PET/CT in lung cancer screening represents a severe challenge for clinical decision-making. This study aimed to develop a clinical-translatable radiomics nomogram for reducing the FPR of PET/CT in lung cancer diagnosis, and to determine the impact of integrating manual diagnosis to the performance of the radiomics nomogram.

Methods

Among 3,947 18F-FDG PET/CT-screened patients with lung lesion, 157 malignant and 111 benign patients were retrospectively enrolled and divided into training and test cohorts. The data of manual diagnosis were recorded. A total of 4,338 features were extracted from CT, thin-section CT, PET and PET/CT, and the four radiomics signatures (RS) were then generated by LASSO method. Radiomics prediction nomogram integrating imaging-based RS and manual diagnosis was developed using multivariable logistic regression. The performances of RS and prediction nomograms were independently validated through key discrimination index and clinical benefit.

Results

The FPR of manual diagnosis was found to be 30.6%. Among the four RS, PET/CT RS exhibited the best performance. By integrating manual diagnosis, the hybrid nomogram integrating PET/CT RS and manual diagnosis demonstrated lowest FPR and highest area under curve (AUC) and Youden index (YI) in both training and test cohorts (FPR: 5.4% and 9.1%, AUC: 0.98 and 0.92, YI: 85.8% and 75.5%, respectively). This hybrid nomogram respectively corrected 78.6% and 37.5% among FPR cases produced by PET/CT RS, without significantly sacrificing its sensitivity. The net benefit of hybrid nomogram appeared highest at <85% threshold probability.

Conclusion

The established hybrid nomogram integrating PET/CT RS and manual diagnosis can significantly reduce FPR, improve diagnostic accuracy and enhance clinical benefit compared to manual diagnosis. By integrating manual diagnosis, the performance of this hybrid nomogram is superior to PET/CT RS, indicating the importance of clinicians’ judgement as an essential information source for improving radiomics diagnostic approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68:394–424.

    PubMed  Google Scholar 

  2. Wood DE, Kazerooni EA, Baum SL, Eapen GA, Ettinger DS, Hou L, et al. Lung Cancer screening, version 3.2018, NCCN clinical practice guidelines in oncology. J Natl Compr Cancer Netw. 2018;16:412–41.

    Google Scholar 

  3. Kim SK, Allen-Auerbach M, Goldin J, Fueger BJ, Dahlbom M, Brown M, et al. Accuracy of PET/CT in characterization of solitary pulmonary lesions. J Nucl Med. 2007;48:214–20.

    PubMed  Google Scholar 

  4. Kagna O, Solomonov A, Keidar Z, Bar-Shalom R, Fruchter O, Yigla M, et al. The value of FDG-PET/CT in assessing single pulmonary nodules in patients at high risk of lung cancer. Eur J Nucl Med Mol Imaging. 2009;36:997–1004.

    PubMed  Google Scholar 

  5. Jeong SY, Lee KS, Shin KM, Bae YA, Kim BT, Choe BK, et al. Efficacy of PET/CT in the characterization of solid or partly solid solitary pulmonary nodules. Lung Cancer. 2008;61:186–94.

    PubMed  Google Scholar 

  6. Deppen SA, Blume JD, Kensinger CD, Morgan AM, Aldrich MC, Massion PP, et al. Accuracy of FDG-PET to diagnose lung cancer in areas with infectious lung disease: a meta-analysis. JAMA. 2014;312:1227–36.

    CAS  PubMed  PubMed Central  Google Scholar 

  7. Maiga AW, Deppen SA, Mercaldo SF, Blume JD, Montgomery C, Vaszar LT, et al. Assessment of Fluorodeoxyglucose F18-labeled positron emission tomography for diagnosis of high-risk lung nodules. JAMA Surg. 2018;153:329–34.

    PubMed  Google Scholar 

  8. Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology. 2016;278:563–77.

    PubMed  Google Scholar 

  9. Shetty N, Noronha V, Joshi A, Rangarajan V, Purandare N, Mohapatra PR, et al. Diagnostic and treatment dilemma of dual pathology of lung cancer and disseminated tuberculosis. J Clin Oncol. 2014;32:e7–9.

    PubMed  Google Scholar 

  10. Glasziou P, Rose P, Heneghan C, Balla J. Diagnosis using "test of treatment". BMJ. 2009;338:b1312.

    PubMed  Google Scholar 

  11. Li Y, Su M, Li F, Kuang A, Tian R. The value of (1)(8)F-FDG-PET/CT in the differential diagnosis of solitary pulmonary nodules in areas with a high incidence of tuberculosis. Ann Nucl Med. 2011;25:804–11.

    PubMed  Google Scholar 

  12. Haroon A, Zumla A, Bomanji J. Role of fluorine 18 fluorodeoxyglucose positron emission tomography-computed tomography in focal and generalized infectious and inflammatory disorders. Clin Infect Dis. 2012;54:1333–41.

    CAS  PubMed  Google Scholar 

  13. Kim IJ, Lee JS, Kim SJ, Kim YK, Jeong YJ, Jun S, et al. Double-phase 18F-FDG PET-CT for determination of pulmonary tuberculoma activity. Eur J Nucl Med Mol Imaging. 2008;35:808–14.

    PubMed  Google Scholar 

  14. Kang F, Wang S, Tian F, Zhao M, Zhang M, Wang Z, et al. Comparing the diagnostic potential of 68Ga-Alfatide II and 18F-FDG in differentiating between non-small cell lung cancer and tuberculosis. J Nucl Med. 2016;57:672–7.

    CAS  PubMed  Google Scholar 

  15. Lambin P, Leijenaar RTH, Deist TM, Peerlings J, de Jong EEC, van Timmeren J, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017;14:749–62.

    PubMed  Google Scholar 

  16. Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RG, Granton P, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012;48:441–6.

    PubMed  PubMed Central  Google Scholar 

  17. Hatt M, Tixier F, Visvikis D, Cheze Le Rest C. Radiomics in PET/CT: more than meets the eye? J Nucl Med. 2017;58:365–6.

    PubMed  Google Scholar 

  18. Aerts H. Data science in radiology: a path forward. Clin Cancer Res. 2018;24:532–4.

    PubMed  Google Scholar 

  19. Bi WL, Hosny A, Schabath MB, Giger ML, Birkbak NJ, Mehrtash A, et al. Artificial intelligence in cancer imaging: clinical challenges and applications. CA Cancer J Clin. 2019;69:127–57.

    PubMed  PubMed Central  Google Scholar 

  20. Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun. 2014;5:4006.

    CAS  PubMed  Google Scholar 

  21. Hochhegger B, Zanon M, Altmayer S, Pacini GS, Balbinot F, Francisco MZ, et al. Advances in imaging and automated quantification of malignant pulmonary diseases: a state-of-the-art review. Lung. 2018;196:633–42.

    PubMed  Google Scholar 

  22. Perandini S, Soardi GA, Motton M, Augelli R, Dallaserra C, Puntel G, et al. Enhanced characterization of solid solitary pulmonary nodules with Bayesian analysis-based computer-aided diagnosis. World J Radiol. 2016;8:729–34.

    PubMed  PubMed Central  Google Scholar 

  23. Ma JC, Wang Q, Ren YC, Hu HB, Zhao J. Automatic lung nodule classification with radiomics approach. SPIE; 2016; https://doi.org/10.1117/12.2220768.

  24. Desseroit MC, Tixier F, Weber WA, Siegel BA, Cheze Le Rest C, Visvikis D, et al. Reliability of PET/CT shape and heterogeneity features in functional and morphologic components of non-small cell lung Cancer tumors: a repeatability analysis in a prospective multicenter cohort. J Nucl Med. 2017;58:406–11.

    CAS  PubMed  PubMed Central  Google Scholar 

  25. Sollini M, Cozzi L, Antunovic L, Chiti A, Kirienko M. PET Radiomics in NSCLC: state of the art and a proposal for harmonization of methodology. Sci Rep. 2017;7:358.

    CAS  PubMed  PubMed Central  Google Scholar 

  26. Hatt M, Laurent B, Fayad H, Jaouen V, Visvikis D, Le Rest CC. Tumour functional sphericity from PET images: prognostic value in NSCLC and impact of delineation method. Eur J Nucl Med Mol Imaging. 2018;45:630–41.

    CAS  PubMed  Google Scholar 

  27. Arshad MA, Thornton A, Lu H, Tam H, Wallitt K, Rodgers N, et al. Discovery of pre-therapy 2-deoxy-2-(18)F-fluoro-D-glucose positron emission tomography-based radiomics classifiers of survival outcome in non-small-cell lung cancer patients. Eur J Nucl Med Mol Imaging. 2019;46:455–66.

    PubMed  PubMed Central  Google Scholar 

  28. Kirienko M, Cozzi L, Antunovic L, Lozza L, Fogliata A, Voulaz E, et al. Prediction of disease-free survival by the PET/CT radiomic signature in non-small cell lung cancer patients undergoing surgery. Eur J Nucl Med Mol Imaging. 2018;45:207–17.

    PubMed  Google Scholar 

  29. Kirienko M, Cozzi L, Rossi A, Voulaz E, Antunovic L, Fogliata A, et al. Ability of FDG PET and CT radiomics features to differentiate between primary and metastatic lung lesions. Eur J Nucl Med Mol Imaging. 2018;45:1649–60.

    PubMed  Google Scholar 

  30. Delbeke D, Coleman RE, Guiberteau MJ, Brown ML, Royal HD, Siegel BA, et al. Procedure guideline for tumor imaging with 18F-FDG PET/CT 1.0. J Nucl Med. 2006;47:885–95.

    PubMed  Google Scholar 

  31. Huang YQ, Liang CH, He L, Tian J, Liang CS, Chen X, et al. Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer. J Clin Oncol. 2016;34:2157–64.

    PubMed  Google Scholar 

  32. Liu Z, Zhang XY, Shi YJ, Wang L, Zhu HT, Tang Z, et al. Radiomics analysis for evaluation of pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Clin Cancer Res. 2017;23:7253–62.

    CAS  PubMed  Google Scholar 

  33. Vickers AJ, Cronin AM, Elkin EB, Gonen M. Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers. BMC Med Inform Decis Mak. 2008;8:53.

    PubMed  PubMed Central  Google Scholar 

  34. Rubin GD, Lyo JK, Paik DS, Sherbondy AJ, Chow LC, Leung AN, et al. Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology. 2005;234:274–83.

    PubMed  Google Scholar 

  35. Gould MK, Ananth L, Barnett PG, Veterans Affairs SCSG. A clinical model to estimate the pretest probability of lung cancer in patients with solitary pulmonary nodules. Chest. 2007;131:383–8.

    PubMed  Google Scholar 

  36. Soardi GA, Perandini S, Motton M, Montemezzi S. Assessing probability of malignancy in solid solitary pulmonary nodules with a new Bayesian calculator: improving diagnostic accuracy by means of expanded and updated features. Eur Radiol. 2015;25:155–62.

    CAS  PubMed  Google Scholar 

  37. Digumarthy SR, Padole AM, Lo Gullo R, Singh R, Shepard JO, Kalra MK. CT texture analysis of histologically proven benign and malignant lung lesions. Medicine (Baltimore). 2018;97:e11172.

    Google Scholar 

  38. Choi W, Oh JH, Riyahi S, Liu CJ, Jiang F, Chen W, et al. Radiomics analysis of pulmonary nodules in low-dose CT for early detection of lung cancer. Med Phys. 2018;45:1537–49.

    PubMed  Google Scholar 

  39. Nie Y, Li Q, Li F, Pu Y, Appelbaum D, Doi K. Integrating PET and CT information to improve diagnostic accuracy for lung nodules: a semiautomatic computer-aided method. J Nucl Med. 2006;47:1075–80.

    PubMed  Google Scholar 

  40. Chen S, Harmon S, Perk T, Li X, Chen M, Li Y, et al. Diagnostic classification of solitary pulmonary nodules using dual time (18)F-FDG PET/CT image texture features in granuloma-endemic regions. Sci Rep. 2017;7:9370.

    PubMed  PubMed Central  Google Scholar 

  41. Miwa K, Inubushi M, Wagatsuma K, Nagao M, Murata T, Koyama M, et al. FDG uptake heterogeneity evaluated by fractal analysis improves the differential diagnosis of pulmonary nodules. Eur J Radiol. 2014;83:715–9.

    PubMed  Google Scholar 

  42. Chen JH, Asch SM. Machine learning and prediction in medicine - beyond the peak of inflated expectations. N Engl J Med. 2017;376:2507–9.

    PubMed  PubMed Central  Google Scholar 

  43. Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ. 2016;352:i6.

    PubMed  PubMed Central  Google Scholar 

  44. Gould MK, Maclean CC, Kuschner WG, Rydzak CE, Owens DK. Accuracy of positron emission tomography for diagnosis of pulmonary nodules and mass lesions: a meta-analysis. JAMA. 2001;285:914–24.

    CAS  PubMed  Google Scholar 

  45. Kim H, Park CM, Gwak J, Hwang EJ, Lee SY, Jung J, et al. Effect of CT Reconstruction Algorithm on the Diagnostic Performance of Radiomics Models: A Task-Based Approach for Pulmonary Subsolid Nodules. AJR Am J Roentgenol. 2019;212:505–12.

    PubMed  Google Scholar 

  46. Vallieres M, Laberge S, Diamant A, El Naqa I. Enhancement of multimodality texture-based prediction models via optimization of PET and MR image acquisition protocols: a proof of concept. Phys Med Biol. 2017;62:8536–65.

    PubMed  Google Scholar 

Download references

Acknowledgments

We would like to thank Zhe Wang, Zhiyong Quan, Ni Wang, Jingwei Yi, Qingju Zhang, Jin Zeng and Xiaohu Zhao for their technical assistance. Fei Kang thanks Prof. Yaochi Yang and Shuangqin Wu for providing the necessary impetus to conduct this study.

Funding

This work was supported by the National Natural Science Foundation of China (Grant Nos. 81871379, 816771713, 81601521), the National Key R&D Program of China (Grant No. 2016YFC0103804), and the Young Elite Scientists Sponsorship Program of China Association for Science and Technology (Grant No. 2017QNRC001).

Author information

Author notes

  1. Fei Kang, Wei Mu and Jie Gong contributed equally to this work.

Authors and Affiliations

  1. Department of Nuclear Medicine, Xijing Hospital, Fourth Military Medical University, No.127 West Changle Road, Xi’an, China

    Fei Kang, Shengjun Wang, Guoquan Li, Guiyu Li & Jing Wang

  2. Key Laboratory of Molecular Imaging of Chinese Academy of Sciences, Institute of Automation, Chinese Academy of Sciences, No. 95 Zhongguancun East Road, Beijing, China

    Wei Mu & Jie Tian

  3. Department of Cancer Physiology, H. Lee Moffitt Cancer Center, Tampa, FL, USA

    Wei Mu

  4. Life Sciences Research Center, School of Life Sciences and Technology, Xidian University, No. 266 Xifeng Road, Xi’an, China

    Jie Gong, Wei Qin & Jie Tian

Authors
  1. Fei Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Mu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shengjun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guoquan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guiyu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Wei Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jie Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Wei Qin, Jie Tian or Jing Wang.

Ethics declarations

Conflict of interest

No other potential conflict of interest relevant to this article was reported.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

This retrospective analysis was approved by the Ethics Committee of Xijing Hospital (Approval No. KY20173008–1), and the informed consent was waived.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Advanced Image Analyses (Radiomics and Artificial Intelligence).

Electronic supplementary material

Esm 1

(XLSX 69 kb)

Esm 2

(DOCX 2.12 MB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, F., Mu, W., Gong, J. et al. Integrating manual diagnosis into radiomics for reducing the false positive rate of 18F-FDG PET/CT diagnosis in patients with suspected lung cancer. Eur J Nucl Med Mol Imaging 46, 2770–2779 (2019). https://doi.org/10.1007/s00259-019-04418-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00259-019-04418-0

Keywords

Search

Navigation