Skip to main content

Advertisement

SpringerLink
Log in

Detecting the location of lung cancer on thoracoscopic images using deep convolutional neural networks

  • Original Article
  • Published:
Surgery Today Aims and scope Submit manuscript

Abstract

Objectives

The prevalence of minimally invasive surgeries has increased the need for tumor detection using thoracoscopic images during lung cancer surgery. We conducted this study to analyze the efficacy of a deep convolutional neural network (DCNN) for tumor detection using recorded thoracoscopic images of pulmonary surfaces.

Materials and methods

We collected 644 intraoperative thoracoscopic images of changes in pulmonary appearance from 427 patients with lung cancer between 2012 and 2021. The lesion areas on the thoracoscopic images were detected by bounding boxes using an advanced version of YOLO, a well-known DCNN for object detection. The DCNN model was trained and evaluated by a 15-fold cross-validation scheme. Each predicted bounding box was considered successful detection when it overlapped more than 50% of the lesion areas annotated by board-certified surgeons.

Results and conclusions

Precision, recall, and F1-measured values of 91.9%, 90.5%, and 91.1%, respectively, were obtained. The presence of lymphatic vessel invasion was associated with successful detection (p = 0.045). The presence of pathological pleural invasion also showed a tendency toward successful detection (p = 0.081). The proposed DCNN-based algorithm yielded an accuracy of more than 90% tumor detection. These algorithms will help surgeons detect lung cancer displayed on a screen automatically.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Arbour KC. Riely GJ Systemic therapy for locally advanced and metastatic non-small cell lung cancer: a review. JAMA. 2019;322:764–74. https://doi.org/10.1001/jama.2019.11058.

    Article  CAS  PubMed  Google Scholar 

  2. Miller M, Hanna N. Advances in systemic therapy for non-small cell lung cancer. BMJ. 2021. https://doi.org/10.1136/bmj.n2363.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Postmus PE, Kerr KM, Oudkerk M, Senan S, Waller DA, Vansteenkiste J, et al. Early and locally advanced non-small-cell lung cancer (NSCLC): ESMO clinical practice guidelines for diagnosis, treatment and follow-up. Ann Oncol. 2017;28:iv1–21. https://doi.org/10.1093/annonc/mdx222.

    Article  CAS  PubMed  Google Scholar 

  4. Ettinger DS, Wood DE, Aisner DL, Akerley W, Bauman JR, Bharat A, et al. NCCN guidelines insights: non-small cell lung cancer, version 2.2021. J Natl Compr Canc Netw. 2021;19:254–66. https://doi.org/10.6004/jnccn.2021.0013.

    Article  CAS  PubMed  Google Scholar 

  5. Maconachie R, Mercer T, Navani N, McVeigh G. Lung cancer: diagnosis and management: summary of updated NICE guidance. BMJ. 2019;364:l1049. https://doi.org/10.1136/bmj.l1049.

    Article  PubMed  Google Scholar 

  6. Louie BE, Wilson JL, Kim S, Cerfolio RJ, Park BJ, Farivar AS, et al. Comparison of video-assisted thoracoscopic surgery and robotic approaches for clinical stage I and stage II non-small cell lung cancer using the Society of Thoracic Surgeons Database. Ann Thorac Surg. 2016;102:917–24. https://doi.org/10.1016/j.athoracsur.2016.03.032.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Ujiie H, Gregor A, Yasufuku K. Minimally invasive surgical approaches for lung cancer. Expert Rev Respir Med. 2019;13:571–8. https://doi.org/10.1080/17476348.2019.1610399.

    Article  CAS  PubMed  Google Scholar 

  8. Lui TK, Tsui VW, Leung WK. Accuracy of artificial intelligence–assisted detection of upper GI lesions: a systematic review and meta-analysis. Gastrointest Endosc. 2020;92:821–30. https://doi.org/10.1016/j.gie.2020.06.034.

    Article  PubMed  Google Scholar 

  9. Paderno A, Holshinger FC, Piazza C. Videomics: bringing deep learning to diagnostic endoscopy. Curr Opin Otolaryngol Head Neck Surg. 2021;29:143–8. https://doi.org/10.1097/MOO.0000000000000697.

    Article  PubMed  Google Scholar 

  10. Zhang R, Zheng Y, Poon CC, Shen D, Lau JY. Polyp detection during colonoscopy using a regression-based convolutional neural network with a tracker. Pattern Recognit. 2018;83:209–19. https://doi.org/10.1016/j.patcog.2018.05.026.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Pacal I, Karaboga D. A robust real-time deep learning based automatic polyp detection system. Comput Biol Med. 2021;134:104519. https://doi.org/10.1016/j.compbiomed.2021.104519.

    Article  PubMed  Google Scholar 

  12. Hirasawa T, Aoyama K, Tanimoto T, Ishihara S, Shichijo S, Ozawa T, et al. Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer. 2018;21:653–60. https://doi.org/10.1007/s10120-018-0793-2.

    Article  PubMed  Google Scholar 

  13. Ohmori M, Ishihara R, Aoyama K, Nakagawa K, Iwagami H, Matsuura N, et al. Endoscopic detection and differentiation of esophageal lesions using a deep neural network. Gastrointest Endosc. 2020;91:301–9. https://doi.org/10.1016/j.gie.2019.09.034.

    Article  PubMed  Google Scholar 

  14. Tamashiro A, Yoshio T, Ishiyama A, Tsuchida T, Hijikata K, Yoshimizu S, et al. Artificial intelligence-based detection of pharyngeal cancer using convolutional neural networks. Dig Endosc. 2020;32:1057–65. https://doi.org/10.1111/den.13653.

    Article  PubMed  Google Scholar 

  15. Quin K, Li J, Fang Y, Xu Y, Wu J, Zhang H, et al. Convolution neural network for the diagnosis of wireless capsule endoscopy: a systematic review and meta-analysis. Surg Endosc. 2022;36:16–31. https://doi.org/10.1007/s00464-021-08689-3.

    Article  Google Scholar 

  16. Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. p. 779–788.

  17. Redmon J, Farhadi A. YOLOv3: an incremental improvement. arXiv:1804.02767 [Preprint]. 2018. Available from: https://doi.org/10.48550/arXiv.1804.02767

  18. Bochkovskiy A, Wang CY, Liao HYM. YOLOv4: optimal speed and accuracy of object detection. arXiv:2004.10934 [Preprint]. 2020. Available from: https://doi.org/10.48550/arXiv.2004.10934

  19. Wang CY, Bochkovskiy A, Liao HYM. Scaled-YOLOv4: scaling cross stage partial network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). 2021. p. 13029–13038.

  20. Zhang J, Xia Y, Cui H, Zhang Y. Pulmonary nodule detection in medical images: a survey. Biomed Signal Process Control. 2018;43:138–47. https://doi.org/10.1016/j.bspc.2018.01.011.

    Article  Google Scholar 

  21. Zhang C, Sun X, Dang K, Li K, Guo XW, Chang J, et al. Toward an expert level of lung cancer detection and classification using a deep convolutional neural network. Oncol. 2019;24:1159–65. https://doi.org/10.1634/theoncologist.2018-0908.

    Article  Google Scholar 

  22. Nolden M, Zelzer S, Seitel A, Wald D, Müller M, Franz AM, et al. The medical imaging interaction toolkit: challenges and advances: 10 years of open-source development. Int J Comput Assist Radiol Surg. 2013;8:607–20. https://doi.org/10.1007/s11548-013-0840-8.

    Article  PubMed  Google Scholar 

  23. Russakovskym O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet large scale visual recognition challenge. Int J Comput Vis. 2015;115:211–52. https://doi.org/10.1007/s11263-015-0816-y.

    Article  Google Scholar 

  24. Tran KA, Kondrashova O, Bradley A, Williams ED, Pearson JV, Waddell N. Deep learning in cancer diagnosis, prognosis and treatment selection. Genome Med. 2021;13:1–17. https://doi.org/10.1186/s13073-021-00968-x.

    Article  Google Scholar 

  25. Kaul V, Enslin S, Gross SA. History of artificial intelligence in medicine. Gastrointest Endosc. 2020;92:807–12. https://doi.org/10.1016/j.gie.2020.06.040.

    Article  PubMed  Google Scholar 

  26. Alexandre LA, Nobre N, Casteleiro J. Color and position versus texture features for endoscopic polyp detection. Proc Int Conf Biomed Eng Inform. 2008;2:38–42. https://doi.org/10.1109/BMEI.2008.246.

    Article  Google Scholar 

  27. Iwahori Y, Hattori A, Adachi Y, Bhuyan MK, Woodham RJ, Kasugai K. Automatic detection of polyp using hessian filter and HOG features. Procedia Comput Sci. 2015;60:730–9. https://doi.org/10.1016/j.procs.2015.08.226.

    Article  Google Scholar 

  28. Chan HP, Samala RK, Hadjiiski LM, Zhou C. Deep learning in medical image analysis. Adv Exp Med Biol. 2020;1213:3–21. https://doi.org/10.1007/978-3-030-33128-3_1.

    Article  PubMed  PubMed Central  Google Scholar 

  29. van der Laak J, Litjens G, Ciompi F. Deep learning in histopathology: the path to the clinic. Nat Med. 2021;27:775–84. https://doi.org/10.1038/s41591-021-01343-4.

    Article  CAS  PubMed  Google Scholar 

Download references

Funding

This research is based on the Cooperative Research Project of Research Center for Biomedical Engineering.

Author information

Authors and Affiliations

  1. Department of Thoracic Surgery, Tokyo Medical and Dental University, Tokyo, Japan

    Yuya Ishikawa & Kenichi Okubo

  2. Department of Biomedical Information, Institute of Biomaterials and Bioengineering, Tokyo Medical and Dental University, 2-3-10, Surugadai, Chiyoda-ku, Tokyo, 101-0062, Japan

    Takaaki Sugino & Yoshikazu Nakajima

Authors
  1. Yuya Ishikawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Takaaki Sugino
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kenichi Okubo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yoshikazu Nakajima
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yoshikazu Nakajima.

Ethics declarations

Conflict of interest

There are no disclosures.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ishikawa, Y., Sugino, T., Okubo, K. et al. Detecting the location of lung cancer on thoracoscopic images using deep convolutional neural networks. Surg Today 53, 1380–1387 (2023). https://doi.org/10.1007/s00595-023-02708-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00595-023-02708-7

Keywords

Search

Navigation