A novel transfer learning approach for the classification of histological images of colorectal cancer

The Journal of Supercomputing (2021)Cite this article

Abstract

Colorectal cancer (CRC) is the second most diagnosed cancer in the United States. It is identified by histopathological evaluations of microscopic images of the cancerous region, relying on a subjective interpretation. The Colorectal Histology dataset used in this study contains 5000 images, made available by the University Medical Center Mannheim. This approach proposes the automatic identification of eight types of tissues found in CRC histopathological evaluation. We apply Transfer Learning from architectures of Convolutional Neural Networks (CNNs). We modify the structures of CNNs to extract features from the images and input them to well-known machine learning methods: Naive Bayes, Multilayer Perceptron, k-Nearest Neighbors, Random Forest, and Support Vector Machine (SVM). We evaluated 108 extractor–classifier combinations. The one that achieved the best results is DenseNet169 with SVM (RBF), reaching an Accuracy of 92.083% and F1-Score of 92.117%. Therefore, our approach is capable of distinguishing tissues found in CRC histopathological evaluation.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Notes

  1. 1.

    We considered the SVM classifier with different kernels as different classifiers.

References

  1. 1.

    Abir F, Alva S, Longo WE, Audiso R, Virgo KS, Johnson FE (2006) The postoperative surveillance of patients with colon cancer and rectal cancer. Am J Surg 192(1):100–108. https://doi.org/10.1016/j.amjsurg.2006.01.053

    Article  Google Scholar 

  2. 2.

    Alhindi TJ, Kalra S, Ng KH, Afrin A, Tizhoosh HR (2018) Comparing LBP, HOG and deep features for classification of histopathology images. In: 2018 International Joint Conference on Neural Networks (IJCNN), IEEE, pp 1–7, https://doi.org/10.1109/IJCNN.2018.8489329

  3. 3.

    Araújo T, Aresta G, Castro E, Rouco J, Aguiar P, Eloy C, Polónia A, Campilho A (2017) Classification of breast cancer histology images using convolutional neural networks. PLoS ONE 12(6):e0177544. https://doi.org/10.1371/journal.pone.0177544

    Article  Google Scholar 

  4. 4.

    Arnold M, Sierra MS, Laversanne M, Soerjomataram I, Jemal A, Bray F (2017) Global patterns and trends in colorectal cancer incidence and mortality. Gut 66(4):683–691. https://doi.org/10.1136/gutjnl-2015-310912

    Article  Google Scholar 

  5. 5.

    Bayramoglu N, Heikkilä J (2016) Transfer learning for cell nuclei classification in histopathology images. In: European Conference on Computer Vision, Springer, pp 532–539

  6. 6.

    Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  Google Scholar 

  7. 7.

    Chatterjee R, Maitra T, Islam SH, Hassan MM, Alamri A, Fortino G (2019) A novel machine learning based feature selection for motor imagery EEG signal classification in internet of medical things environment. Future Gener Comput Syst 98:419–434. https://doi.org/10.1016/j.future.2019.01.048

    Article  Google Scholar 

  8. 8.

    Choi JY, Yoo TK, Seo JG, Kwak J, Um TT, Rim TH (2017) Multi-categorical deep learning neural network to classify retinal images: a pilot study employing small database. PLoS ONE. https://doi.org/10.1371/journal.pone.0187336

    Article  Google Scholar 

  9. 9.

    Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1251–1258. https://doi.org/10.1109/CVPR.2017.195

  10. 10.

    Coudray N, Ocampo PS, Sakellaropoulos T, Narula N, Snuderl M, Fenyö D, Moreira AL, Razavian N, Tsirigos A (2018) Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med 24(10):1559–1567. https://doi.org/10.1038/s41591-018-0177-5

    Article  Google Scholar 

  11. 11.

    da Nóbrega RVM, Filho PPR, Rodrigues MB, da Silva SPP, Júnior CMJMD, de Albuquerque VHC (2018) Lung nodule malignancy classification in chest computed tomography images using transfer learning and convolutional neural networks. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3895-1

    Article  Google Scholar 

  12. 12.

    Fukunaga K, Narendra PM (1975) A branch and bound algorithm for computing k-nearest neighbors. IEEE Trans Comput 24(7):750–753. https://doi.org/10.1109/T-C.1975.224297

    Article  MATH  Google Scholar 

  13. 13.

    Gecer B, Aksoy S, Mercan E, Shapiro LG, Weaver DL, Elmore JG (2018) Detection and classification of cancer in whole slide breast histopathology images using deep convolutional networks. Pattern Recogn 84:345–356. https://doi.org/10.1016/j.patcog.2018.07.022

    Article  Google Scholar 

  14. 14.

    Girard L, Rodriguez-Canales J, Behrens C, Thompson DM, Botros IW, Tang H, Xie Y, Rekhtman N, Travis WD, Wistuba II et al (2016) An expression signature as an aid to the histologic classification of non-small cell lung cancer. Clin Cancer Res 22(19):4880–4889. https://doi.org/10.1158/1078-0432

    Article  Google Scholar 

  15. 15.

    Han D, Liu Q, Fan W (2018) A new image classification method using CNN transfer learning and web data augmentation. Expert Syst Appl 95:43–56. https://doi.org/10.1016/j.eswa.2017.11.028

    Article  Google Scholar 

  16. 16.

    Han T, Nunes VX, De Freitas Souza LF, Marques AG, Silva ICL, Junior MAAF, Sun J, Filho PPR (2020) Internet of medical things-based on deep learning techniques for segmentation of lung and stroke regions in CT scans. IEEE Access 8:71117–71135. https://doi.org/10.1109/ACCESS.2020.2987932

    Article  Google Scholar 

  17. 17.

    Hassan MM, Ullah S, Hossain MS, Alelaiwi A (2020) An end-to-end deep learning model for human activity recognition from highly sparse body sensor data in internet of medical things environment. J Supercomput. https://doi.org/10.1007/s11227-020-03361-4

    Article  Google Scholar 

  18. 18.

    Haykin S (2009) Neural networks and learning machines, vol 3. Pearson Upper Saddle River, NJ, USA

    Google Scholar 

  19. 19.

    He K, Zhang X, Ren S, Sun J (2016a) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  20. 20.

    He K, Zhang X, Ren S, Sun J (2016b) Identity mappings in deep residual networks. In: European Conference on Computer Vision, Springer, pp 630–645

  21. 21.

    Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:170404861

  22. 22.

    Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4700–4708. https://doi.org/10.1109/CVPR.2017.243

  23. 23.

    Hussein S, Cao K, Song Q, Bagci U (2017) Risk stratification of lung nodules using 3d CNN-based multi-task learning. Lecture Notes in Computer Science. Springer, Cham, pp 249–260

    Google Scholar 

  24. 24.

    Jr Dourado CM, da Silva SP, da Nóbrega RV, Barros AC, Reboucas Filho PP, de Albuquerque VHC (2019) Deep learning IoT system for online stroke detection in skull computed tomography images. Comput Netw 152:25–39. https://doi.org/10.1016/j.comnet.2019.01.019

    Article  Google Scholar 

  25. 25.

    Kather J, Weis CA, Bianconi F, Melchers S, Schad L, Gaiser T, Marx A, Zöllner F (2016a) Multi-class texture analysis in colorectal cancer histology. Sci Rep 6:27988. https://doi.org/10.1038/srep27988

    Article  Google Scholar 

  26. 26.

    Kather JN, Zöllner FG, Bianconi F, Melchers SM, Schad LR, Gaiser T, Marx A, Weis CA (2016b) Collection of textures in colorectal cancer histology. https://doi.org/10.5281/ZENODO.53169

  27. 27.

    Kather JN, Krisam J, Charoentong P, Luedde T, Herpel E, Weis CA, Gaiser T, Marx A, Valous NA, Ferber D et al (2019) Predicting survival from colorectal cancer histology slides using deep learning: a retrospective multicenter study. PLoS Med. https://doi.org/10.1371/journal.pmed.1002730

    Article  Google Scholar 

  28. 28.

    Kermany DS, Goldbaum M, Cai W, Valentim CC, Liang H, Baxter SL, McKeown A, Yang G, Wu X, Yan F, Dong J, Prasadha MK, Pei J, Ting MY, Zhu J, Li C, Hewett S, Dong J, Ziyar I, Shi A, Zhang R, Zheng L, Hou R, Shi W, Fu X, Duan Y, Huu VA, Wen C, Zhang ED, Zhang CL, Li O, Wang X, Singer MA, Sun X, Xu J, Tafreshi A, Lewis MA, Xia H, Zhang K (2018) Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5):1122–1131.e9. https://doi.org/10.1016/j.cell.2018.02.010

    Article  Google Scholar 

  29. 29.

    Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:14126980

  30. 30.

    Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  31. 31.

    Loh BCS, Then PHH (2017) Deep learning for cardiac computer-aided diagnosis: benefits, issues & solutions. mHealth 3:45. https://doi.org/10.21037/mhealth.2017.09.01

    Article  Google Scholar 

  32. 32.

    Marques JAL, Cortez PC, Madeiro JPDV, Fong SJ, Schlindwein FS, De Albuquerque VHC (2019) Automatic cardiotocography diagnostic system based on hilbert transform and adaptive threshold technique. IEEE Access 7:73085–73094

    Article  Google Scholar 

  33. 33.

    Marques JAL, Cortez PC, Madeiro JP, de Albuquerque VHC, Fong SJ, Schlindwein FS (2020) Nonlinear characterization and complexity analysis of cardiotocographic examinations using entropy measures. J Supercomput 76(2):1305–1320

    Article  Google Scholar 

  34. 34.

    Olivares R, Munoz R, Soto R, Crawford B, Cárdenas D, Ponce A, Taramasco C (2020) An optimized brain-based algorithm for classifying parkinson’s disease. Appl Sci 10(5):1827. https://doi.org/10.3390/app10051827

    Article  Google Scholar 

  35. 35.

    Orenstein EC, Beijbom O (2017) Transfer learning and deep feature extraction for planktonic image data sets. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, pp 1082–1088. https://doi.org/10.1109/WACV.2017.125

  36. 36.

    Paul R, Hawkins SH, Balagurunathan Y, Schabath MB, Gillies RJ, Hall LO, Goldgof DB (2016) Deep feature transfer learning in combination with traditional features predicts survival among patients with lung adenocarcinoma. Tomography 2(4):388. https://doi.org/10.18383/j.tom.2016.00211

    Article  Google Scholar 

  37. 37.

    Rachapudi V, Devi GL (2020) Improved convolutional neural network based histopathological image classification. Evolut Intell 284:1–7

    Google Scholar 

  38. 38.

    Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252. https://doi.org/10.1007/s11263-015-0816-y

    MathSciNet  Article  Google Scholar 

  39. 39.

    Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4510–4520

  40. 40.

    Santos MA, Munoz R, Olivares R, Rebouças Filho PP, Del Ser J, de Albuquerque VHC (2020) Online heart monitoring systems on the internet of health things environments: a survey, a reference model and an outlook. Inf Fusion 53:222–239. https://doi.org/10.1016/j.inffus.2019.06.004

    Article  Google Scholar 

  41. 41.

    Sarmento RM, Vasconcelos FFX, Rebouças Filho PP, Wu W, De Albuquerque VHC (2019) Automatic neuroimage processing and analysis in stroke—a systematic review. IEEE Rev Biomed Eng 13:130–155

    Article  Google Scholar 

  42. 42.

    Sarmento RM, Vasconcelos FF, Filho PPR, de Albuquerque VHC (2020) An IoT platform for the analysis of brain CT images based on parzen analysis. Future Gener Comput Syst 105:135–147. https://doi.org/10.1016/j.future.2019.11.033

    Article  Google Scholar 

  43. 43.

    Shaban M, Awan R, Fraz MM, Azam A, Tsang YW, Snead D, Rajpoot NM (2020) Context-aware convolutional neural network for grading of colorectal cancer histology images. IEEE Trans Med Imaging. https://doi.org/10.1109/tmi.2020.2971006

    Article  Google Scholar 

  44. 44.

    Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298. https://doi.org/10.1109/TMI.2016.2528162

    Article  Google Scholar 

  45. 45.

    Siegel RL, Miller KD, Fedewa SA, Ahnen DJ, Meester RG, Barzi A, Jemal A (2017) Colorectal cancer statistics, 2017. CA Cancer J Clin 67(3):177–193. https://doi.org/10.3322/caac.21395

    Article  Google Scholar 

  46. 46.

    Siegel RL, Miller KD, Sauer AG, Fedewa SA, Butterly LF, Anderson JC, Cercek A, Smith RA, Jemal A (2020) Colorectal cancer statistics, 2020. Cancer J Clin CA. https://doi.org/10.3322/caac.21601

    Article  Google Scholar 

  47. 47.

    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556

  48. 48.

    Slaoui M, Fiette L (2010) Histopathology procedures: from tissue sampling to histopathological evaluation. Methods in molecular biology. Humana Press, Totowa, pp 69–82

    Google Scholar 

  49. 49.

    Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1–9. https://doi.org/10.1109/CVPR.2015.7298594

  50. 50.

    Theodoridis S, Koutroumbas K (2008) Pattern recognition, 4th edn. Academic Press, USA

    Google Scholar 

  51. 51.

    Tseng KK, Zhang R, Chen CM, Hassan MM (2020) DNetUnet: a semi-supervised CNN of medical image segmentation for super-computing AI service. J Supercomput. https://doi.org/10.1007/s11227-020-03407-7

    Article  Google Scholar 

  52. 52.

    Vapnik VN (1998) Statistical learning theory. Wiley-Interscience, Hoboken

    Google Scholar 

  53. 53.

    Wang C, Shi J, Zhang Q, Ying S (2017) Histopathological image classification with bilinear convolutional neural networks. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, pp 4050–4053

  54. 54.

    Wang EK, Chen CM, Hassan MM, Almogren A (2020) A deep learning based medical image segmentation technique in internet-of-medical-things domain. Future Gener Comput Syst 108:135–144. https://doi.org/10.1016/j.future.2020.02.054

    Article  Google Scholar 

  55. 55.

    Wang P, Hu X, Li Y, Liu Q, Zhu X (2016) Automatic cell nuclei segmentation and classification of breast cancer histopathology images. Signal Process 122:1–13. https://doi.org/10.1016/j.sigpro.2015.11.011

    Article  Google Scholar 

  56. 56.

    Xu J, Xiang L, Liu Q, Gilmore H, Wu J, Tang J, Madabhushi A (2015) Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images. IEEE Trans Med Imaging 35(1):119–130. https://doi.org/10.1109/TMI.2015.2458702

    Article  Google Scholar 

  57. 57.

    Xu Y, Jia Z, Wang LB, Ai Y, Zhang F, Lai M, Eric I, Chang C (2017) Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features. BMC Bioinform 18(1):281. https://doi.org/10.1186/s12859-017-1685-x

    Article  Google Scholar 

  58. 58.

    Yu KH, Zhang C, Berry GJ, Altman RB, Ré C, Rubin DL, Snyder M (2016) Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nat Commun 7:12474. https://doi.org/10.1038/ncomms12474

    Article  Google Scholar 

  59. 59.

    Yu KH, Wang F, Berry GJ, Ré C, Altman RB, Snyder M, Kohane IS (2020) Classifying non-small cell lung cancer types and transcriptomic subtypes using convolutional neural networks. J Am Med Inform Assoc 27(5):757–769. https://doi.org/10.1093/jamia/ocz230

    Article  Google Scholar 

  60. 60.

    Zoph B, Le QV (2016) Neural architecture search with reinforcement learning. arXiv preprint arXiv:161101578

Download references

Acknowledgements

The authors are grateful to the Deanship of Scientific Research at King Saud University for funding this work through Vice Deanship of Scientific Research Chairs: Chair of Pervasive and Mobile Computing. This study was also supported in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Also, Pedro Pedrosa Rebouças Filho acknowledges the sponsorship from the Brazilian National Council for Research and Development (CNPq) via Grants Nos. 431709/2018-1 and 311973/2018-3.

Author information

Affiliations

  1. Laboratório de Processamento de Imagens, Sinais e Computação Aplicada, Instituto Federal do Ceará, Fortaleza, CE, Brazil

    Elene Firmeza Ohata, João Victor Souza das Chagas, Gabriel Maia Bezerra, Victor Hugo Costa de Albuquerque & Pedro Pedrosa Rebouças Filho

  2. College of Computer and Information Sciences and Research Chair of Pervasive and Mobile Computing, King Saud University, Riyadh, 11543, Saudi Arabia

    Mohammad Mehedi Hassan

  3. Department of Computer Science, Federal Institute of Education, Science and Technology of Ceará, Fortaleza, CE, 60040-215, Brazil

    Victor Hugo Costa de Albuquerque & Pedro Pedrosa Rebouças Filho

  4. Department of Teleinformatics Engineering, Federal University of Ceará, Fortaleza, CE, 60020-181, Brazil

    Victor Hugo Costa de Albuquerque & Pedro Pedrosa Rebouças Filho

Authors
  1. Elene Firmeza Ohata
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. João Victor Souza das Chagas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gabriel Maia Bezerra
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohammad Mehedi Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Victor Hugo Costa de Albuquerque
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Pedro Pedrosa Rebouças Filho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Victor Hugo Costa de Albuquerque.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ohata, E.F., Chagas, J.V.S.d., Bezerra, G.M. et al. A novel transfer learning approach for the classification of histological images of colorectal cancer. J Supercomput (2021). https://doi.org/10.1007/s11227-020-03575-6

Download citation

Keywords

  • Transfer learning
  • Colorectal cancer
  • Histological images
  • Convolutional neural networks