DeepLayout: A Semantic Segmentation Approach to Page Layout Analysis

  • Yixin Li
  • Yajun Zou
  • Jinwen MaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10956)


In this paper, we present DeepLayout, a new approach to page layout analysis. Previous work divides the problem into unsupervised segmentation and classification. Instead of a step-wise method, we adopt semantic segmentation which is an end-to-end trainable deep neural network. Our proposed segmentation model takes only document image as input and predicts per pixel saliency maps. For the post-processing part, we use connected component analysis to restore the bounding boxes from the prediction map. The main contribution is that we successfully bring RLSA into our post-processing procedures to specify the boundaries. The experimental results on ICDAR2017 POD competition dataset show that our proposed page layout analysis algorithm achieves good mAP score, outperforms most of other competition participants.


Page layout analysis Document segmentation Document image understanding Semantic segmentation and deep learning 



This work was supported by the Natural Science Foundation of China for Grant 61171138.


  1. 1.
    Yi, X., Gao, L., Liao, Y., et al.: CNN based page object detection in document images. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 230–235. IEEE (2017)Google Scholar
  2. 2.
    Cesarini, F., Lastri, M., Marinai, S., et al.: Encoding of modified X-Y trees for document classification. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 1131–1136. IEEE (2001)Google Scholar
  3. 3.
    Priyadharshini, N., Vijaya, M.S.: Document segmentation and region classification using multilayer perceptron. Int. J. Comput. Sci. Issues 10(2 part 1), 193 (2013)Google Scholar
  4. 4.
    Lin, M.W., Tapamo, J.R., Ndovie, B.: A texture-based method for document segmentation and classification. S. Afr. Comput. J. 36, 49–56 (2006)Google Scholar
  5. 5.
    Chen, K., Yin, F., Liu, C.L.: Hybrid page segmentation with efficient whitespace rectangles extraction and grouping. In: International Conference on Document Analysis and Recognition, pp. 958–962. IEEE Computer Society (2013)Google Scholar
  6. 6.
    Ren, S., He, K., Girshick, R., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)CrossRefGoogle Scholar
  7. 7.
    Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). Scholar
  8. 8.
    Chen, L.C., Papandreou, G., Kokkinos, I., et al.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)CrossRefGoogle Scholar
  9. 9.
    Gao, L., Yi, X., Jiang, Z., et al.: Competition on page object detection. In: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR 2017, pp. 1417–1422. IEEE (2017)Google Scholar
  10. 10.
    Chu, W.T., Liu, F.: Mathematical formula detection in heterogeneous document images. In: Technologies and Applications of Artificial Intelligence, pp. 140–145. IEEE (2014)Google Scholar
  11. 11.
    Gao, L., Yi, X., Liao, Y., et al.: A deep learning-based formula detection method for PDF documents. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 553–558. IEEE (2017)Google Scholar
  12. 12.
    Hassan, T., Baumgartner, R.: Table recognition and understanding from PDF files. In: International Conference on Document Analysis and Recognition, pp. 1143–1147. IEEE (2007)Google Scholar
  13. 13.
    Oyedotun, O.K., Khashman, A.: Document segmentation using textural features summarization and feedforward neural network. Appl. Intell. 45(1), 198–212 (2016)CrossRefGoogle Scholar
  14. 14.
    Yang, X., Yumer, E., Asente, P., et al.: Learning to extract semantic structure from documents using multimodal fully convolutional neural networks. arXiv preprint arXiv:1706.02337 (2017)
  15. 15.
    Shotton, J., Fitzgibbon, A., Cook, M., et al.: Real-time human pose recognition in parts from single depth images. In: Computer Vision and Pattern Recognition, pp. 1297–1304. IEEE (2011)Google Scholar
  16. 16.
    Ciresan, D., Giusti, A., Gambardella, L.M., et al.: Deep neural networks segment neuronal membranes in electron microscopy images. Adv. Neural. Inf. Process. Syst. 2843–2851 (2012)Google Scholar
  17. 17.
    Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640 (2017)CrossRefGoogle Scholar
  18. 18.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  19. 19.
    He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  20. 20.
    Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. In: International Conference on Learning Representations (2016). arXiv:1511.07122
  21. 21.
    Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). Scholar
  22. 22.
    Koltun, V.: Efficient inference in fully connected CRFs with Gaussian edge potentials. In: International Conference on Neural Information Processing Systems, pp. 109–117. Curran Associates Inc. (2011)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Information Science, School of Mathematical Sciences and LMAMPeking UniversityBeijingChina

Personalised recommendations