A Deep Learning Based Pipeline for Efficient Oral Cancer Screening on Whole Slide Images

  • Jiahao Lu
  • Nataša Sladoje
  • Christina Runow Stark
  • Eva Darai Ramqvist
  • Jan-Michaél Hirsch
  • Joakim LindbladEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12132)


Oral cancer incidence is rapidly increasing worldwide. The most important determinant factor in cancer survival is early diagnosis. To facilitate large scale screening, we propose a fully automated pipeline for oral cancer detection on whole slide cytology images. The pipeline consists of fully convolutional regression-based nucleus detection, followed by per-cell focus selection, and CNN based classification. Our novel focus selection step provides fast per-cell focus decisions at human-level accuracy. We demonstrate that the pipeline provides efficient cancer classification of whole slide cytology images, improving over previous results both in terms of accuracy and feasibility. The complete source code is made available as open source (


CNN Whole slide imaging Big data Cytology Detection Focus selection Classification 


  1. 1.
    Campanella, G., et al.: Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25(8), 1301–1309 (2019)CrossRefGoogle Scholar
  2. 2.
    Cruz-Roa, A., et al.: Accurate and reproducible invasive breast cancer detection in whole-slide images: a deep learning approach for quantifying tumor extent. Sci. Rep. 7, 46450 (2017)CrossRefGoogle Scholar
  3. 3.
    Deroulers, C., Ameisen, D., Badoual, M., Gerin, C., Granier, A., Lartaud, M.: Analyzing huge pathology images with open source software. Diagnostic Pathol. 8(1), 92 (2013)CrossRefGoogle Scholar
  4. 4.
    Falk, T., et al.: U-Net: deep learning for cell counting, detection, and morphometry. Nat. Methods 16(1), 67–70 (2019)CrossRefGoogle Scholar
  5. 5.
    Farahani, N., Parwani, A., Pantanowitz, L.: Whole slide imaging in pathology: advantages, limitations, and emerging perspectives. Pathol. Lab Med. Int. 7, 23–33 (2015)Google Scholar
  6. 6.
    Ferzli, R., Karam, L.: A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB). IEEE Trans. Image Process. 18(4), 717–728 (2009)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Girshick, R.: Fast R-CNN. arXiv:1504.08083 [cs], September 2015
  8. 8.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. arXiv:1311.2524 [cs], October 2014
  9. 9.
    Guan, J., Zhang, W., Gu, J., Ren, H.: No-reference blur assessment based on edge modeling. J. Vis. Commun. Image Represent. 29, 1–7 (2015)CrossRefGoogle Scholar
  10. 10.
    Gupta, A., et al.: Deep learning in image cytometry: a review. Cytometry Part A 95(4), 366–380 (2019)CrossRefGoogle Scholar
  11. 11.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. arXiv:1512.03385 [cs], December 2015
  12. 12.
    Höfener, H., Homeyer, A., Weiss, N., Molin, J., Lundström, C., Hahn, H.: Deep learning nuclei detection: a simple approach can deliver state-of-the-art results. Comput. Med. Imag. Graph. 70, 43–52 (2018)CrossRefGoogle Scholar
  13. 13.
    Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. arXiv:1608.06993 [cs], January 2018
  14. 14.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167 [cs] (2015)
  15. 15.
    Jabalee, J., et al.: Identification of malignancy-associated changes in histologically normal tumor-adjacent epithelium of patients with HPV-positive oropharyngeal cancer. Anal. Cellular Pathol. 2018, 1–9 (2018)CrossRefGoogle Scholar
  16. 16.
    Kainz, P., Urschler, M., Schulter, S., Wohlhart, P., Lepetit, V.: You should use regression to detect cells. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 276–283. Springer, Cham (2015). Scholar
  17. 17.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 [cs], December 2014
  18. 18.
    Korbar, B., et al.: Deep learning for classification of colorectal polyps on whole-slide images. J. Pathol. Inform. 8, 30 (2017)CrossRefGoogle Scholar
  19. 19.
    Narvekar, N.D., Karam, L.J.: A no-reference image blur metric based on the cumulative probability of blur detection (CPBD). IEEE Trans. Image Process. 20(9), 2678–2683 (2011)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of IEEE Conference on CVPR, pp. 779–788 (2016)Google Scholar
  21. 21.
    Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: Proceedings of IEEE Conference on CVPR, pp. 7263–7271 (2017)Google Scholar
  22. 22.
    Redmon, J., Farhadi, A.: YOLOv3: An Incremental Improvement. arXiv:1804.02767 [cs], April 2018
  23. 23.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. arXiv:1506.01497 [cs], January 2016
  24. 24.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W., Frangi, A. (eds.) Medical Image Computing and Computer Assisted Intervention MICCAI 2015, pp. 234–241. Springer, Cham (2015). Scholar
  25. 25.
    Sankaranarayanan, R., et al.: Long term effect of visual screening on oral cancer incidence and mortality in a randomized trial in Kerala. India. Oral Oncol. 49(4), 314–321 (2013)CrossRefGoogle Scholar
  26. 26.
    Speight, P., et al.: Screening for oral cancer—a perspective from the global oral cancer forum. Oral Surg., Oral Med. Oral Pathol. Oral Radiol. 123(6), 680–687 (2017)CrossRefGoogle Scholar
  27. 27.
    Stewart, B., Wild, C.P., et al.: World cancer report 2014. Public Health (2014)Google Scholar
  28. 28.
    Teramoto, A., et al.: Automated classification of benign and malignant cells from lung cytological images using deep convolutional neural network. Inform. Med. Unlocked 16, 100205 (2019)CrossRefGoogle Scholar
  29. 29.
    Us-Krasovec, M., et al.: Malignancy associated changes in epithelial cells of buccal mucosa: a potential cancer detection test. Anal. Quantit. Cytol. Histol. 27(5), 254–262 (2005)Google Scholar
  30. 30.
    Warnakulasuriya, S.: Global epidemiology of oral and oropharyngeal cancer. Oral Oncol. 45(4–5), 309–316 (2009)CrossRefGoogle Scholar
  31. 31.
    Wetzer, E., Gay, J., Harlin, H., Lindblad, J., Sladoje, N.: When texture matters: texture-focused CNNs outperform general data augmentation and pretraining in oral cancer detection. In: Proceedings of IEEE International Symposium on Biomedical Imaging (ISBI) (2020) forthcomingGoogle Scholar
  32. 32.
    Wieslander, H., et al.: Deep convolutional neural networks for detecting cellular changes due to malignancy. In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 82–89. IEEE, October 2017Google Scholar
  33. 33.
    Xie, W., Noble, J., Zisserman, A.: Microscopy cell counting and detection with fully convolutional regression networks. Comput. Methods Biomech. Biomed. Eng. Imag. Visual. 6(3), 283–292 (2018)CrossRefGoogle Scholar
  34. 34.
    Xie, Y., Xing, F., Kong, X., Su, H., Yang, L.: Beyond classification: structured regression for robust cell detection using convolutional neural network. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 358–365. Springer, Cham (2015). Scholar
  35. 35.
    Zhang, L., Lu, L., Nogues, I., Summers, R.M., Liu, S., Yao, J.: DeepPap: deep convolutional networks for cervical cell classification. IEEE J. Biomed. Health Inform. 21(6), 1633–1643 (2017)CrossRefGoogle Scholar
  36. 36.
    Zou, Z., Shi, Z., Guo, Y., Ye, J.: Object detection in 20 years: a survey. arXiv:1905.05055 [cs], May 2019

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Centre for Image Analysis, Department of ITUppsala UniversityUppsalaSweden
  2. 2.Department of Orofacial Medicine at SödersjukhusetFolktandvården Stockholms Län ABStockholmSweden
  3. 3.Department of Clinical Pathology and CytologyKarolinska University HospitalStockholmSweden
  4. 4.Department of Surgical sciencesUppsala UniversityUppsalaSweden

Personalised recommendations