Segmentation of cervical cells for automated screening of cervical cancer: a review

  • Abid SarwarEmail author
  • Abrar Ali Sheikh
  • Jatinder Manhas
  • Vinod Sharma


In automated screening of cervical cytology, the morphological features of cell play a determining role. To avoid false diagnosis, urgent need of precise extraction of these features led to emergence of new segmentation models. In this paper author aspire to present literature review of research done in the field of segmentation stage in automatic screening of cervical smear images. Total of 78 publications are considered for the time period of 40 years. A detailed study of segmentation technique proposed in each publication is considered, which presents a chronological development and up-gradation of segmentation models. This review assist researcher to have thorough knowledge of various state-of-art segmentation models and the problems and complexities required to be tackled, for unambiguous determination of malignancies in cervical cytology.


Segmentation Cervical cancer Pap smear Nucleus Cytoplasm 



  1. Abikoye OC et al (2017) A K-means and fuzzy logic-based system for clinical diagnosis (staging) of cervical cancer. Int J Telemed Clin Pract 2(2):168–196Google Scholar
  2. Ashok B, Aruna P (2016) Comparison of Feature selection methods for diagnosis of cervical cancer using SVM classifier. Int J Eng Res Appl 6:94–99Google Scholar
  3. Athinarayanan S, Srinath MV (2016) Classification of cervical cancer cells in PAP smear screening test. ICTACT J Image Video Process 6(4):1234–1238Google Scholar
  4. Bak E, Kayvan N, Brockway JP (2004) Efficient segmentation framework of cell images in noise environments. In: The 26th annual international conference of the IEEE engineering in medicine and biology society, vol 1. IEEEGoogle Scholar
  5. Bamford P, Brian L (1998) Bayesian analysis of cell nucleus segmentation by a Viterbi search based active contour. In: Proceedings of fourteenth international conference on pattern recognition (Cat. No. 98EX170), vol 1. IEEEGoogle Scholar
  6. Bamford P, Lovell B (1998) Unsupervised cell nucleus segmentation with active contours. Sig Process 71(2):203–213zbMATHGoogle Scholar
  7. Bergmeir C, Silvente MG, Benítez JM (2012) Segmentation of cervical cell nuclei in high-resolution microscopic images: a new algorithm and a web-based software framework. Comput Methods Programs Biomed 107(3):497–512Google Scholar
  8. Cahn RL, Poulsen RS, Toussaint G (1977) Segmentation of cervical cell images. J Histochem Cytochem 25(7):681–688Google Scholar
  9. Chankong T, Theera-Umpon N, Auephanwiriyakul S (2014) Automatic cervical cell segmentation and classification in Pap smears. Comput Methods Programs Biomed 113(2):539–556Google Scholar
  10. Chaturvedi A, Gillison ML (2010) Human papillomavirus and head and neck cancer. In: Epidemiology, pathogenesis, and prevention of head and neck cancer. Springer, New York, NY, pp 87–116Google Scholar
  11. Chen Y-F et al (2014) Semi-automatic segmentation and classification of pap smear cells. IEEE J Biomed Health Inf 18(1):94–108MathSciNetGoogle Scholar
  12. Chuanyun X, Yang Z, Sen W (2013) Cell segmentation in cervical smear images using polar coordinates GVF snake with radiating edge map. J Multimed 8:213–219Google Scholar
  13. Duth PS (2015) A fast and robust level set method for medical image segmentation. Int J Appl Eng Res 10(11):28645–28655Google Scholar
  14. Fan J, Li S, Zhang C (2013a) Color cell image segmentation based on Chan-Vese model for vector-valued images. J Softw Eng Appl 6(10):554Google Scholar
  15. Fan J et al (2013b) A separating algorithm for overlapping cell images. J Softw Eng Appl 6(04):179Google Scholar
  16. Ferri F, Pudil P, Hatef M, Kittler J (1994) Comparative study of techniques for large-scale feature selection. In Gelsma E, Kamal L (eds) Pattern recognition in practice IV. Elsevier Science, pp 403–413.Google Scholar
  17. Garrido A, De La Blanca NP (2000) Applying deformable templates for cell image segmentation. Pattern Recognit 33(5):821–832Google Scholar
  18. Gençtav A, Aksoy S, Önder S (2012) Unsupervised segmentation and classification of cervical cell images. Pattern Recognit 45(12):4151–4168Google Scholar
  19. Ghafar R et al. (2003) Segmentation of stretched pap smear cytology images using clustering algorithm. In: Proceedings of world congress on medical physics and biomedical engineering, vol 4, no 2356Google Scholar
  20. GLOBOCAN (2002) database: summary table by cancer. Archived from the original on 16 June 2008Google Scholar
  21. Göçeri E (2016) Fully automated liver segmentation using Sobolev gradient-based level set evolution. Int J Numer Methods Biomed Eng 32(11):e02765Google Scholar
  22. Goceri E, Numan G (2017) Deep learning in medical image analysis: recent advances and future trends. In: International conferences computer graphics, visualization, computer vision and image processingGoogle Scholar
  23. Göçeri E, Gürcan MN, Dicle O (2014) Fully automated liver segmentation from SPIR image series. Comput Biol Med 53:265–278Google Scholar
  24. Göçeri E, Ünlü MZ, Dicle O (2015) A comparative performance evaluation of various approaches for liver segmentation from SPIR images. Turkish J Electr Eng Comput Sci 23(3):741–768Google Scholar
  25. Happy SL, Chatterjee S, Sheet D (2015) Unsupervised segmentation of overlapping cervical cell cytoplasm. arXiv preprint arXiv:1505.05601
  26. Holmquist J et al (1978) Computer analysis of cervical cells. Automatic feature extraction and classification. J Histochem Cytochem 26(11):1000–1017Google Scholar
  27. Human Papillomavirus (HPV) and Cervical cancer: Fact sheet, WHO (2016)Google Scholar
  28. Isa NAM (2005) Automated edge detection technique for Pap smear images using moving K-means clustering and modified seed based region growing algorithm. Int J Comput Internet Manag 13(3):45–59Google Scholar
  29. Jantzen J, Norup J, Dounias G, Bjerregaard B (2005) Pap-smear benchmark data for pattern classification. Nature inspired Smart Information Systems (NiSIS 2005), 1–9.Google Scholar
  30. Jusman Y et al. (2014) Intelligent screening systems for cervical cancer. Sci World J 2014:810368Google Scholar
  31. Kale A, Aksoy S (2010) Segmentation of cervical cell images. In: Proceedings of the 2010 20th international conference on pattern recognition. IEEE Computer SocietyGoogle Scholar
  32. Kent A (2010) HPV vaccination and testing. Rev ObstetrGynecol 3(1):33e4Google Scholar
  33. Kong H, Gurcan M, Belkacem-Boussaid K (2011) Partitioning histopathological images: an integrated framework for supervised color-texture segmentation and cell splitting. IEEE Trans Med Imaging 30(9):1661–1677Google Scholar
  34. Lee H, Kim J (2016) Segmentation of overlapping cervical cells in microscopic images with superpixel partitioning and cell-wise contour refinement. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshopsGoogle Scholar
  35. Li K et al (2012) Cytoplasm and nucleus segmentation in cervical smear images using Radiating GVF Snake. Pattern Recognit 45(4):1255–1264Google Scholar
  36. Lin C-H, Chan Y-K, Chen C-C (2009) Detection and segmentation of cervical cell cytoplast and nucleus. Int J Imaging Syst Technol 19(3):260–270Google Scholar
  37. Lu Z, Carneiro G, Bradley AP (2013) Automated nucleus and cytoplasm segmentation of overlapping cervical cells. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, HeidelbergGoogle Scholar
  38. Lu Z et al (2016) Evaluation of three algorithms for the segmentation of overlapping cervical cells.”. IEEE J Biomed Health Inf 21(2):441–450Google Scholar
  39. Luck BL et al (2005) An image model and segmentation algorithm for reflectance confocal images of in vivo cervical tissue. IEEE Trans Image Process 14(9):1265–1276Google Scholar
  40. Mahanta LB et al (2011) Fuzzy mathematical and shape theoretic approach to cervical cell classification. Int J Comput Appl 975:8887Google Scholar
  41. Martin E (2003) Pap-smear classification. Master’s Thesis, Technical University of Denmark, Oersted- DTU, AutomationGoogle Scholar
  42. Mehnert A et al. (2014) A structural texture approach for characterising malignancy associated changes in pap smears based on mean-shift and the watershed transform. In: 2014 22nd international conference on pattern recognition. IEEEGoogle Scholar
  43. Meiquan X et al. (2018) Cervical cytology intelligent diagnosis based on object detection technology. 1st Conference on Medical Imaging with Deep Learning (MIDL 2018), Amsterdam, The Netherlands (2018)Google Scholar
  44. Melo JCB, Cavalcanti GDC, Guimaraes KS (2003) PCA feature extraction for protein structure prediction. In: Proceedings of the international joint conference on neural networks, vol 4. IEEE, 2003Google Scholar
  45. Moshavegh R et al. (2012) Automated segmentation of free-lying cell nuclei in pap smears for malignancy-associated change analysis. In: 2012 annual international conference of the IEEE engineering in medicine and biology society. IEEEGoogle Scholar
  46. Munoz N et al (2003) Epidemiologic classification of human papillomavirus types associated with cervical cancer. N Engl J Med 348(6):518–527Google Scholar
  47. Nedzved A, Ablameyko S, Pitas I (2000) Morphological segmentation of histology cell images. In: Proceedings 15th international conference on pattern recognition. ICPR-2000, vol 1. IEEEGoogle Scholar
  48. Niraimathi M, Vellaichamy S (2015) Comparison of segmentation algorithms by a mathematical model for resolving islands and gulfs in nuclei of cervical cell images. Int Arab J Inf Technol 12(5)Google Scholar
  49. Nosrati MS, Ghassan H (2015) Segmentation of overlapping cervical cells: A variational method with star-shape prior. In: 2015 IEEE 12th international symposium on biomedical imaging (ISBI). IEEEGoogle Scholar
  50. Nosrati MS, Hamarneh G (2014) A variational approach for overlapping cell segmentation. In: ISBI overlapping cervical cytology image segmentation challenge, vol 2014, pp 1–2Google Scholar
  51. Oscanoa J, Mena M, Kemper G (2015) A detection method of ectocervical cell nuclei for pap test images, based on adaptive thresholds and local derivatives. Int J Multimed Ubiquitous Eng 10(2):37–50Google Scholar
  52. Pai P-Y, Chang C-C, Chan Y-K (2012) Nucleus and cytoplast contour detector from a cervical smear image. Expert Syst Appl 39(1):154–161Google Scholar
  53. Phoulady HA et al. (2015) An approach for overlapping cell segmentation in multi-layer cervical cell volumes. In: The second overlapping cervical cytology image segmentation challenge-IEEE ISBIGoogle Scholar
  54. Phoulady HA et al. (2016) A new approach to detect and segment overlapping cells in multi-layer cervical cell volume images. In: 2016 IEEE 13th international symposium on biomedical imaging (ISBI). IEEEGoogle Scholar
  55. Plissiti ME et al. (2009) Automated detection of cell nuclei in PAP stained cervical smear images using fuzzy clustering. In: 4th European conference of the international federation for medical and biological engineering. Springer, Berlin, HeidelbergGoogle Scholar
  56. Plissiti ME, Nikou C (2011) Cell nuclei segmentation by learning a physically based deformable model. In: 2011 17th international conference on digital signal processing (DSP). IEEEGoogle Scholar
  57. Plissiti ME, Nikou C (2012) Overlapping cell nuclei segmentation using a spatially adaptive active physical model. IEEE Trans Image Process 21(11):4568–4580MathSciNetzbMATHGoogle Scholar
  58. Plissiti ME, Nikou C, Charchanti A (2010) Watershed-based segmentation of cell nuclei boundaries in Pap smear images. In: Proceedings of the 10th IEEE international conference on information technology and applications in biomedicine. IEEEGoogle Scholar
  59. Plissiti ME, Nikou C, Charchanti A (2011a) Combining shape, texture and intensity features for cell nuclei extraction in Pap smear images. Pattern Recognit Lett 32(6):838–853Google Scholar
  60. Plissiti ME, Nikou C, Charchanti A (2011b) Automated detection of cell nuclei in pap smear images using morphological reconstruction and clustering. IEEE Trans Inf Technol Biomed 15(2):233–241Google Scholar
  61. Poulsen RS, Ilario P (1995) Region of interest finding in reduced resolution colour imagery—application to cancer cell detection. Pattern Recognit 28(11):1645–1655Google Scholar
  62. Poulsen RS et al (1977) High resolution analysis of cervical cells–a progress report. J Histochem Cytochem 25(7):689–695Google Scholar
  63. Ramesh BV, Raghunandan S, Ramakrishnan KR (1993) Zero crossing edge detection and contour tracing for segmentation of cervical cell nucleus. Def Sci J 43(3):223Google Scholar
  64. Riana D et al. (2014) Color canals modification with canny edge detection and morphological reconstruction for cell nucleus segmentation and area measurement in normal Pap smear images. In: AIP conference proceedings, vol 1589, no 1. AIPGoogle Scholar
  65. Rodenacker K, Bengtsson E (2003) A feature set for cytometry on digitized microscopic images. Anal Cell Pathol 25(1):1–36Google Scholar
  66. Sarbortova H (2013) Final project report detection of cervical cancer in pap smear imagesGoogle Scholar
  67. Savitha B, Subashini P (2013) An adaptive threshold segmentation for detection of nuclei in cervical cells using wavelet shrinkage algorithms. In: Third international conference on computer science, engineering &applications (ICCSEA-2013), vol 10Google Scholar
  68. Schilling T et al (2007) Towards rapid cervical cancer diagnosis: automated detection and classification of pathologic cells in phase-contrast images. Int J Gynecol Cancer 17(1):118–126Google Scholar
  69. Shidham VB et al (2011) p16INK4a immunocytochemistry on cell blocks as an adjunct to cervical cytology: potential reflex testing on specially prepared cell blocks from residual liquid-based cytology specimens. Cytojournal 8:1Google Scholar
  70. Smeulders AWM et al (1978) An image segmentation approach to the analysis of nuclear texture. Acta Histochem 20:217–222Google Scholar
  71. Smeulders AW et al (1979) Texture analysis of cervical cell nuclei by segmentation of chromatin patterns. J Histochem Cytochem 27(1):199–203Google Scholar
  72. Sokouti B, Haghipour S, Tabrizi AD (2012) A pilot study on image analysis techniques for extracting early uterine cervix cancer cell features. J Med Syst 36(3):1901–1907Google Scholar
  73. Song Y et al (2015) Accurate segmentation of cervical cytoplasm and nuclei based on multiscale convolutional network and graph partitioning. IEEE Trans Biomed Eng 62(10):2421–2433Google Scholar
  74. Sulaiman SN et al. (2010) Overlapping cells separation method for cervical cell images. In: 2010 10th international conference on intelligent systems design and applications. IEEEGoogle Scholar
  75. Supriyanto E et al. (2011) Automatic detection system of cervical cancer cells using color intensity classification. Recent Res Comput SciGoogle Scholar
  76. Talukdar J, Nath CK, Talukdar PH (2013) Fuzzy clustering based image segmentation of pap smear images of cervical cancer cell using FCM algorithm. markers 3(1):460–462Google Scholar
  77. Tareef A et al (2017) Automatic segmentation of overlapping cervical smear cells based on local distinctive features and guided shape deformation. Neurocomputing 221:94–107Google Scholar
  78. Tsai M-H et al (2008) Nucleus and cytoplast contour detector of cervical smear image. Pattern Recognit Lett 29(9):1441–1453Google Scholar
  79. Ushizima DM, Bianchi AGC, Carneiro CM (2015) Segmentation of subcellular compartments combining superpixel representation with voronoi diagrams. In: Overlapping cervical cytology image segmentation challenge-IEEE ISBI, vol 2014, pp 1–2Google Scholar
  80. Walker RF et al. (1994) Classification of cervical cell nuclei using morphological segmentation and textural feature extraction. In: Proceedings of ANZIIS’94-Australian New Zealnd intelligent information systems conference. IEEEGoogle Scholar
  81. World Health Organization (2006) Fact sheet no. 297: cancer. February. Retrieved 01 Dec 2007Google Scholar
  82. Xu C, Prince JL (1998) Generalized gradient vector flow external forces for active contours. Signal Process 71(2):131–139zbMATHGoogle Scholar
  83. Xue Z et al. (2010) Automatic extraction of mosaic patterns in uterine cervix images. In: 2010 IEEE 23rd international symposium on computer-based medical systems (CBMS). IEEEGoogle Scholar
  84. Zhang L et al. (2011) A practical segmentation method for automated screening of cervical cytology. In: 2011 international conference on intelligent computation and bio-medical instrumentation. IEEEGoogle Scholar
  85. Zhang L, Kong H, Chin CT, Liu S, Chen Z, Wang T, Chen S (2014) Segmentation of cytoplasm and nuclei of abnormal cells in cervical cytology using global and local graph cuts. Comput Med Imaging Graph 38(5):369–380Google Scholar
  86. Zhang J et al (2016) Segmentation of overlapping cells in cervical smears based on spatial relationship and overlapping translucency light transmission model. Pattern Recognit 60:286–295Google Scholar
  87. Zhang L et al (2017) Graph-based segmentation of abnormal nuclei in cervical cytology. Comput Med Imaging Graph 56:38–48Google Scholar
  88. Zhao L et al (2016) Automatic cytoplasm and nuclei segmentation for color cervical smear image using an efficient gap-search MRF. Comput Biol Med 71:46–56Google Scholar
  89. Zinser GERHARD, Komitowski DYMITR (1983) Segmentation of cell nuclei in tissue section analysis. J Histochem Cytochem 31(1):94–100Google Scholar
  90. Zongker D, Anil J (1996) Algorithms for feature selection: an evaluation. In: Proceedings of 13th international conference on pattern recognition, vol 2. IEEEGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  • Abid Sarwar
    • 1
    Email author
  • Abrar Ali Sheikh
    • 1
  • Jatinder Manhas
    • 1
  • Vinod Sharma
    • 1
  1. 1.Department of Computer Sc. & ITUniversity of JammuJammu and KashmirIndia

Personalised recommendations