Skip to main content

Advertisement

SpringerLink
Log in

Spotting malignancies from gastric endoscopic images using deep learning

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Gastric cancer is a common kind of malignancies, with yearly occurrences exceeding one million worldwide in 2017. Typically, ulcerous and cancerous tissues develop abnormal morphologies through courses of progression. Endoscopy is a routinely adopted means for examination of gastrointestinal tract for malignancy. Early and timely detection of malignancy closely correlate with good prognosis. Repeated presentation of similar frames from gastrointestinal tract endoscopy often weakens attention for practitioners to result in true patients missed out to incur higher medical cost and unnecessary morbidity. Highly needed is an automatic means for spotting visual abnormality and prompts for attention for medical staff for more thorough examination.

Methods

We conduct classification of benign ulcer and cancer for gastrointestinal endoscopic color images using deep neural network and transfer-learning approach. Using clinical data gathered from Gil Hospital, we built a dataset comprised of 200 normal, 367 cancer, and 220 ulcer cases, and applied the inception, ResNet, and VGGNet models pretrained on ImageNet. Three classes were defined—normal, benign ulcer, and cancer, and three separate binary classifiers were built—those for normal vs cancer, normal vs ulcer, and cancer vs ulcer for the corresponding classification tasks. For each task, considering inherent randomness entailed in the deep learning process, we performed data partitioning and model building experiments 100 times and averaged the performance values.

Results

Areas under curves of respective receiver operating characteristics were 0.95, 0.97, and 0.85 for the three classifiers. The ResNet showed the highest level of performance. The cases involving normal, i.e., normal vs ulcer and normal vs cancer resulted in accuracies above 90%. The case of ulcer vs cancer classification resulted in a lower accuracy of 77.1%, possibly due to smaller difference in appearance than those cases involving normal.

Conclusions

The overall level of performance of the proposed method was very promising to encourage applications in clinical environments. Automatic classification using deep learning technique as proposed can be used to complement manual inspection efforts for practitioners to minimize dangers of missed out positives resulting from repetitive sequence of endoscopic frames and weakening attentions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. World Cancer Report 2014 (2014) World Health Organization

  2. http://globocan.iarc.fr. Accessed 23 Jan 2019

  3. Watanabe K, Nagata N, Shimbo T, Nakashima R, Furuhata E, Sakurai T, Akazawa N, Yokoi C, Kobayakawa M, Akiyama J, Mizokami M, Uemura N (2013) Accuracy of endoscopic diagnosis of Helicobacter pylori infection according to level of endoscopic experience and the effect of training. BMC Gastroenterol 13:128

    Article  Google Scholar 

  4. Menon S, Trudgill N (2014) How commonly is upper gastrointestinal cancer missed at endoscopy? A meta-analysis. Endosc Int Open 2(2):E46–E50

    Article  Google Scholar 

  5. Almadi MA, Sewitch M, Barkun AN (2015) Adenoma detection rates decline with increasing procedural hours in an endoscopist’s workload. Can J Gastroenterol Hepatol 29(6):304–308

    Article  Google Scholar 

  6. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

    Article  CAS  Google Scholar 

  7. Miyaki R, Yoshida S, Tanaka S, Kominami Y, Sanomura Y, Matsuo T, Oka S, Raytchev B, Tamaki T, Koide T, Kaneda K, Yoshihara M, Chayama K (2015) A computer system to be used with laser-based endoscopy for quantitative diagnosis of early gastric cancer. J Clin Gastroenterol 49(2):108–115

    Article  Google Scholar 

  8. Zhang X, Hu W, Chen F, Liu J, Yang Y, Wang L, Duan H, Si J (2017) Gastric precancerous diseases classification using CNN with a concise model. PLoS ONE 12(9):e0185508

    Article  Google Scholar 

  9. Zhu R, Zhang R, Xue D (2015) Gastric precancerous diseases classification using CNN with a concise model. In: 2015 8th International Congress on Image and Signal Processing (CISP) 14–16 Oct

  10. Billah M, Waheed S, Rahman MM (2017) An automatic gastrointestinal polyp detection system in video endoscopy using fusion of color wavelet and convolutional neural network features. Int J Biomed Imaging. https://doi.org/10.1155/2017/9545920

    Article  PubMed  PubMed Central  Google Scholar 

  11. Byrne MF, Chapados N, Soudan F, Oertel C, Linares PM, Kelly R, Iqbal N, Chandelier F, Rex DK (2017) Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut 68:94–100

    Article  Google Scholar 

  12. Tajbakhsh N (2015) Automatic polyp detection in colonoscopy videos using an ensemble of convolutional neural networks. In: IEEE 12th International Symposium on Biomedical Imaging (ISBI)

  13. Li B, Meng MQ (2009) Computer-aided detection of bleeding regions for capsule endoscopy images. IEEE Trans Biomed Eng 56(4):1032–1039

    Article  Google Scholar 

  14. Tamai N, Saito Y, Sakamoto T, Nakajima T, Matsuda T, Sumiyama K, Tajiri H, Koyama R, Kido S (2017) Effectiveness of computer-aided diagnosis of colorectal lesions using novel software for magnifying narrow-band imaging: a pilot study. Endosc Int Open 5(8):E690–E694

    Article  Google Scholar 

  15. Clements LM, Kockelman KM (2017) Economic effects of automated vehicles. J Transp Res Board 2606:2606–2614

    Article  Google Scholar 

  16. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Driessche G, Schrittwieser J (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529:484–489

    Article  CAS  Google Scholar 

  17. Gerhardus D (2003) Robot-assisted surgery: the future is here. J Healthc Manag 48(4):242–251

    Article  Google Scholar 

  18. Maroulis DE, Savelonas MA, Karkanis SA, Iakovidis DK, Dimitropoulos N (2005) Computer-aided thyroid nodule detection in ultrasound images. Proceedings of the 18th IEEE symposium on computer-based medical systems (CBMS’05)

  19. Pizer SM, Amburn EP, Austin JD, Cromartie R, Geselowitz A, Greer T, Romeny B, Zimmerman JB, Zuiderveld K (1987) Adaptive histogram equalization and its variations. Comput Vis Graph Image Process 39(3):355–368

    Article  Google Scholar 

  20. Zuiderveld K (1994) Contrast limited adaptive histogram equalization. Graphics gems IV. Academic Press Professional, Inc., San Diego, pp 474–485

    Google Scholar 

  21. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) ImageNet: a large-scale hierarchical image database. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp. 248–255

  22. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. NIPS 1:1097–1105

    Google Scholar 

  23. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1–9

  24. He K, Zhang C, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778

  25. Canziani A, Paszke A, Culurciello E (2016) An analysis of deep neural network models for practical applications. ArXiv:1605.07678

  26. Dauphin GMY, Glorot X, Rifai S, Bengio Y, Goodfellow I, Lavoie E, Muller X, Desjardins G, Warde-Farley D, Vincent P, Courville A, Bergstra J (2012) Unsupervised and transfer learning challenge: a deep learning approach. JMLR 27:97–110

    Google Scholar 

  27. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115–118

    Article  CAS  Google Scholar 

  28. Pogorelov K, Randel KR, Griwodz C, Eskeland SL, de Lange T, Johansen D, Spampinato C, Nguyen DTD, Lux M, Schmidt PT, Riegler M, Halvorsen P (2017) Kvasir: a multi-class image dataset for computer aided gastrointestinal disease detection. In: Proceedings of the 8th ACM on Multimedia Systems Conference, Pages 164–169

Download references

Acknowledgements

This work was supported by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIT) (2018-2-00861, Intelligent SW Technology Development for Medical Data Analysis) and the Gachon University Gil Medical Center (Grant No: 2018-5283). Authors Kim KG and Chung JW equally contributed to this work.

Funding

The authors state that this work has not received any funding.

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, College of Medicine, Gachon University, 38 3-dockjeomro, Namdong-gu, Incheon, 21565, South Korea

    Jang Hyung Lee, Young Jae Kim, Yoon Woo Kim, Sungjin Park & Kwang Gi Kim

  2. Department of Gastroenterology, Gachon-Gil Hospital, College of Medicine, Gachon University, Incheon, South Korea

    Youn-i Choi, Yoon Jae Kim, Dong Kyun Park & Jun-Won Chung

  3. Department of Gastroenterology, Gil Medical Center, School of Medicine, Gachon University, 38 3-dockjeomro, Namdong-gu, Incheon, 21565, South Korea

    Jun-Won Chung

Authors
  1. Jang Hyung Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Young Jae Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yoon Woo Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sungjin Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Youn-i Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yoon Jae Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dong Kyun Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Kwang Gi Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jun-Won Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Kwang Gi Kim or Jun-Won Chung.

Ethics declarations

Disclosure

Authors Jang Hyung Lee, Young Jae Kim, Yoon Woo Kim, Sungjin Park, Yoon Yi Choi, Yoon Jae Kim, Dong Kyun Park, Kwang Gi Kim, and Jun-Won Chung have no financial arrangement or affiliation with any product or services used or discussed in this paper, nor any potential bias against another product or service. The authors declare that there are no conflicts of interest regarding the publication of this paper.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, J.H., Kim, Y.J., Kim, Y.W. et al. Spotting malignancies from gastric endoscopic images using deep learning. Surg Endosc 33, 3790–3797 (2019). https://doi.org/10.1007/s00464-019-06677-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-019-06677-2

Keywords

Search

Navigation