Advertisement

Nuclei Perception Network for Pathology Image Analysis

  • Haojun Xu
  • Yan Gao
  • Liucheng Hu
  • Jie Li
  • Xinbo GaoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11935)

Abstract

Nuclei segmentation is a challenge task in medical image analysis. A digital microscopic tissue image may contain hundreds or even thousands nuclear. Its morphological information provides the biological basis for the diagnosis and classification of diseases. The task requires to detect every nuclear of cells in a densely packed scene and get the segmentation of them for further pathological analysis. Nuclei segmentation can also be described as an instance segmentation task in densely packed scene. In this article, we propose a novel anchor-free dense instance segmentation framework to alleviate the issues. The network detects nuclears and segment them simultaneously. Then the nuclear segmentation mask is aggregated as nuclear instance guided by the offset map generated from the network. The network works by combining target location with pixel-by-pixel classification to distinguish crowded objects. The proposed method performs well on nuclear segmentation dataset.

Keywords

Deep learning CNN Nuclei segmentation 

Notes

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant 61432014, 61772402, U1605252 and 61671339, and in part by National High-Level Talents Special Support Program of China under Grant CS31117200001.

References

  1. 1.
    Kumar, N., Verma, R., Sharma, S., Bhargava, S., Vahadane, A., Sethi, A.: A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans. Med. Imaging 36(7), 1550–1560 (2017)CrossRefGoogle Scholar
  2. 2.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  3. 3.
    Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., Saltz, J.H.: Patch-based convolutional neural network for whole slide tissue image classification. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2016Google Scholar
  4. 4.
    Sheikhzadeh, F., Carraro, A., Korbelik, J., MacAulay, C., Guillaud, M., Ward, R.K.: Automatic labeling of molecular biomarkers on a cell-by-cell basis in immunohistochemistry images using convolutional neural networks. In: Gurcan, M.N. Madabhushi, A. (eds.) Medical Imaging 2016: Digital Pathology. SPIE, March 2016Google Scholar
  5. 5.
    Greenspan, H., van Ginneken, B., Summers, R.M.: Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans. Med. Imaging 35(5), 1153–1159 (2016)CrossRefGoogle Scholar
  6. 6.
    Irshad, H., Veillard, A., Roux, L., Racoceanu, D.: Methods for nuclei detection, segmentation, and classification in digital histopathology: a review—current status and future potential. IEEE Rev. Biomed. Eng. 7, 97–114 (2014)CrossRefGoogle Scholar
  7. 7.
    Wang, C., Shi, J., Zhang, Q., Ying, S.: Histopathological image classification with bilinear convolutional neural networks. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, July 2017Google Scholar
  8. 8.
    Zhang, Y., Yang, L., Chen, J., Fredericksen, M., Hughes, D.P., Chen, D.Z.: Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 408–416. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-66179-7_47CrossRefGoogle Scholar
  9. 9.
    Naylor, P., Lae, M., Reyal, F., Walter, T.: Segmentation of nuclei in histopathology images by deep regression of the distance map. IEEE Trans. Med. Imaging 38(2), 448–459 (2019)CrossRefGoogle Scholar
  10. 10.
    Chen, H., Qi, X., Yu, L., Heng, P.-A.: DCAN: deep contour-aware networks for accurate gland segmentation. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2016Google Scholar
  11. 11.
    Song, J., Xiao, L., Molaei, M., Lian, Z.: Multi-layer boosting sparse convolutional model for generalized nuclear segmentation from histopathology images. Knowl.-Based Syst. 176, 40–53 (2019)CrossRefGoogle Scholar
  12. 12.
    He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 346–361. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10578-9_23CrossRefGoogle Scholar
  13. 13.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)CrossRefGoogle Scholar
  14. 14.
    He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, October 2017Google Scholar
  15. 15.
    Yang, Z., Liu, S., Hu, H., Wang, L., Lin, S.: RepPoints: point set representation for object detection. arXiv preprint arXiv:1904.11490 (2019)
  16. 16.
    Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, July 2017Google Scholar
  17. 17.
    Xu, Y., et al.: Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features. BMC Bioinform. 18(1), 281 (2017)CrossRefGoogle Scholar
  18. 18.
    Chen, J.-M., Li, Y., Jun, X., Gong, L., Wang, L.-W., Liu, W.-L., Liu, J.: Computer-aided prognosis on breast cancer with hematoxylin and eosin histopathology images: a review. Tumor Biol. 39(3), 101042831769455 (2017)CrossRefGoogle Scholar
  19. 19.
    Ali, H.R., et al.: Lymphocyte density determined by computational pathology validated as a predictor of response to neoadjuvant chemotherapy in breast cancer: secondary analysis of the ARTemis trial. Ann. Oncol. 28(8), 1832–1835 (2017)CrossRefGoogle Scholar
  20. 20.
    Xing, F., Yang, L.: Robust nucleus/cell detection and segmentation in digital pathology and microscopy images: a comprehensive review. IEEE Rev. Biomed. Eng. 9, 234–263 (2016)CrossRefGoogle Scholar
  21. 21.
    Cheng, J., Rajapakse, J.C.: Segmentation of clustered nuclei with shape markers and marking function. IEEE Trans. Biomed. Eng. 56(3), 741–748 (2009)CrossRefGoogle Scholar
  22. 22.
    Faridi, P., Danyali, H., Helfroush, M.S., Jahromi, M.A.: An automatic system for cell nuclei pleomorphism segmentation in histopathological images of breast cancer. In: 2016 IEEE Signal Processing in Medicine and Biology Symposium (SPMB). IEEE, December 2016Google Scholar
  23. 23.
    Wang, P., Hu, X., Li, Y., Liu, Q., Zhu, X.: Automatic cell nuclei segmentation and classification of breast cancer histopathology images. Sig. Process. 122, 1–13 (2016)CrossRefGoogle Scholar
  24. 24.
    Filipczuk, P., Kowal, M., Obuchowicz, A.: Automatic breast cancer diagnosis based on k-means clustering and adaptive thresholding hybrid segmentation. In: Choraś, R.S. (ed.) Advances in Intelligent and Soft Computing, vol. 120, pp. 295–302. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-23154-4_33CrossRefGoogle Scholar
  25. 25.
    Gao, Y., et al.: Hierarchical nucleus segmentation in digital pathology images. In: Gurcan, M.N., Madabhushi, A. (eds.) Medical Imaging 2016: Digital Pathology. SPIE, March 2016Google Scholar
  26. 26.
    Guo, P., Evans, A., Bhattacharya, P.: Segmentation of nuclei in digital pathology images. In: 2016 IEEE 15th International Conference on Cognitive Informatics and Cognitive Computing (ICCI* CC), pp. 547–550. IEEE (2016)Google Scholar
  27. 27.
    Al-Kofahi, Y., Lassoued, W., Lee, W., Roysam, B.: Improved automatic detection and segmentation of cell nuclei in histopathology images. IEEE Trans. Biomed. Eng. 57(4), 841–852 (2010)CrossRefGoogle Scholar
  28. 28.
    Rother, C., Kolmogorov, V., Blake, A.: GrabCut. ACM Trans. Graph. 23(3), 309 (2004)CrossRefGoogle Scholar
  29. 29.
    Ali, H.R., et al.: Computational pathology of pre-treatment biopsies identifies lymphocyte density as a predictor of response to neoadjuvant chemotherapy in breast cancer. Breast Cancer Res. 18(1), 21 (2016)CrossRefGoogle Scholar
  30. 30.
    Liao, M., et al.: Automatic segmentation for cell images based on bottleneck detection and ellipse fitting. Neurocomputing 173, 615–622 (2016)CrossRefGoogle Scholar
  31. 31.
    Kharma, N., et al.: Automatic segmentation of cells from microscopic imagery using ellipse detection. IET Image Proc. 1(1), 39 (2007)CrossRefGoogle Scholar
  32. 32.
    Hai, S., Xing, F., Lee, J.D., Peterson, C.A., Yang, L.: Automatic myonuclear detection in isolated single muscle fibers using robust ellipse fitting and sparse representation. IEEE/ACM Trans. Comput. Biol. Bioinf. 11(4), 714–726 (2014)CrossRefGoogle Scholar
  33. 33.
    Veta, M., van Diest, P.J., Kornegoor, R., Huisman, A., Viergever, M.A., Pluim, J.P.W.: Automatic nuclei segmentation in H&E stained breast cancer histopathology images. PLoS One 8(7), e70221 (2013)CrossRefGoogle Scholar
  34. 34.
    Qu, A., et al.: Two-step segmentation of hematoxylin-eosin stained histopathological images for prognosis of breast cancer. In: 2014 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, November 2014Google Scholar
  35. 35.
    Zhang, D., et al.: Panoptic segmentation with an end-to-end cell R-CNN for pathology image analysis. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 237–244. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00934-2_27CrossRefGoogle Scholar
  36. 36.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Computer Vision and Pattern Recognition (2014)Google Scholar
  37. 37.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59(2), 167–181 (2004)CrossRefGoogle Scholar
  38. 38.
    Girshick, R.: Fast r-CNN. In: 2015 IEEE International Conference on Computer Vision (ICCV). IEEE, December 2015Google Scholar
  39. 39.
    Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2016Google Scholar
  40. 40.
    Liu, W., et al.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46448-0_2CrossRefGoogle Scholar
  41. 41.
    Dai, J., Li, Y., He, K., Sun, J.: R-FCN: object detection via region-based fully convolutional networks. In: Advances in Neural Information Processing Systems, pp. 379–387 (2016)Google Scholar
  42. 42.
    Cai, Z., Vasconcelos, N.: Cascade R-CNN: delving into high quality object detection. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, June 2018Google Scholar
  43. 43.
    Fu, C.-Y., Liu, W., Ranga, A., Tyagi, A., Berg, A.C.: DSSD: deconvolutional single shot detector. arXiv preprint arXiv:1701.06659 (2017)
  44. 44.
    Leibe, B., Seemann, E., Schiele, B.: Pedestrian detection in crowded scenes. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005). IEEE (2005)Google Scholar
  45. 45.
    Huang, C., Ai, H., Li, Y., Lao, S.: High-performance rotation invariant multiview face detection. IEEE Trans. Pattern Anal. Mach. Intell. 29(4), 671–686 (2007)CrossRefGoogle Scholar
  46. 46.
    Zhou, Y., Ye, Q., Qiu, Q., Jiao, J.: Oriented response networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, July 2017Google Scholar
  47. 47.
    Law, H., Deng, J.: CornerNet: detecting objects as paired keypoints. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 765–781. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01264-9_45CrossRefGoogle Scholar
  48. 48.
    Dai, J., He, K., Sun, J.: Instance-aware semantic segmentation via multi-task network cascades. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2016Google Scholar
  49. 49.
    Li, Y., Qi, H., Dai, J., Ji, X., Wei, Y.: Fully convolutional instance-aware semantic segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, July 2017Google Scholar
  50. 50.
    Liu, S., Qi, L., Qin, H., Shi, J., Jia, J.: Path aggregation network for instance segmentation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, June 2018Google Scholar
  51. 51.
    Huang, L., Yang, Y., Deng, Y., Yu, Y.: DenseBox: unifying landmark localization with end to end object detection. arXiv preprint arXiv:1509.04874 (2015)
  52. 52.
    Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, July 2017Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Haojun Xu
    • 1
  • Yan Gao
    • 1
  • Liucheng Hu
    • 1
  • Jie Li
    • 1
  • Xinbo Gao
    • 1
    • 2
    Email author
  1. 1.School of Electronic EngineeringXidian UniversityXi’anChina
  2. 2.State Key Laboratory of integrated Services NetworksXi’anChina

Personalised recommendations