Journal of Computer Science and Technology

, Volume 34, Issue 1, pp 155–169 | Cite as

Learning to Generate Posters of Scientific Papers by Probabilistic Graphical Models

  • Yu-Ting Qiang
  • Yan-Wei Fu
  • Xiao Yu
  • Yan-Wen GuoEmail author
  • Zhi-Hua Zhou
  • Leonid Sigal
Regular Paper


Researchers often summarize their work in the form of scientific posters. Posters provide a coherent and efficient way to convey core ideas expressed in scientific papers. Generating a good scientific poster, however, is a complex and time-consuming cognitive task, since such posters need to be readable, informative, and visually aesthetic. In this paper, for the first time, we study the challenging problem of learning to generate posters from scientific papers. To this end, a data-driven framework, which utilizes graphical models, is proposed. Specifically, given content to display, the key elements of a good poster, including attributes of each panel and arrangements of graphical elements, are learned and inferred from data. During the inference stage, the maximum a posterior (MAP) estimation framework is employed to incorporate some design principles. In order to bridge the gap between panel attributes and the composition within each panel, we also propose a recursive page splitting algorithm to generate the panel layout for a poster. To learn and validate our model, we collect and release a new benchmark dataset, called NJU-Fudan Paper-Poster dataset, which consists of scientific papers and corresponding posters with exhaustively labelled panels and attributes. Qualitative and quantitative results indicate the effectiveness of our approach.


graphical design layout automation probabilistic graphical model 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

11390_2019_1904_MOESM1_ESM.pdf (1.2 mb)
ESM 1 (PDF 1207 kb)


  1. [1]
    Jahanian A, Liu J, Tretter D R, Lin Q, Damera-Venkata N, O’Brien-Strain E, Lee S, Fan J, Allebach J P. Automatic design of magazine covers. In Proc. IS&T/SPIE Electronic Imaging, International Society for Optics and Photonics, January 2012, Article ID. 83020.Google Scholar
  2. [2]
    Hunter A, Slatter D, Greig D. Web-based magazine design for self publishers. In Proc. IS&T/SPIE Electronic Imaging, International Society for Optics and Photonics, January 2011, Article ID. 787902.Google Scholar
  3. [3]
    O’Donovan P, Agarwala A, Hertzmann A. Learning layouts for single-page graphic designs. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(8): 1200-1213.CrossRefGoogle Scholar
  4. [4]
    Geigel J, Loui A. Using genetic algorithms for album page layouts. IEEE Multimedia, 2003, 10(4): 16-27.CrossRefGoogle Scholar
  5. [5]
    Yu L F, Yeung S K, Tang C K, Terzopoulos D, Chan T F, Osher S J. Make it home: Automatic optimization of furniture arrangement. ACM Transactions on Graphics, 2011, 30(4): Article No. 86.Google Scholar
  6. [6]
    Merrell P, Schkufza E, Li Z, Agrawala M, Koltun V. Interactive furniture layout using interior design guidelines. ACM Transactions on Graphics, 2011, 30(4): Article No. 87.Google Scholar
  7. [7]
    Cao Y, Lau R W, Chan A B. Look over here: Attentiondirecting composition of manga elements. ACM Transactions on Graphics, 2014, 33(4): Article No. 94.Google Scholar
  8. [8]
    Hurst N, Li W, Marriott K. Review of automatic document formatting. In Proc. the 9th ACM Symposium on Document Engineering, September 2009, pp.99-108.Google Scholar
  9. [9]
    Knuth D E, Plass M F. Breaking paragraphs into lines. Software: Practice and Experience, 1981, 11(11): 1119-1184.zbMATHGoogle Scholar
  10. [10]
    Peels A J H, Janssen N J M, Nawijn W. Document architecture and text formatting. ACM Transactions on Information Systems, 1985, 3(4): 347-369.CrossRefGoogle Scholar
  11. [11]
    Damera-Venkata N, Bento J, O’Brien-Strain E. Probabilistic document model for automated document composition. In Proc. the 11th ACM Symposium on Document Engineering, September 2011, pp.3-12.Google Scholar
  12. [12]
    Mihalcea R, Tarau P. TextRank: Bringing order into text. In Proc. the 2004 Conference on Empirical Methods in Natural Language Processing, July 2004, pp.404-411.Google Scholar
  13. [13]
    Qiang Y T, Fu Y W, Zhou Y W, Zhou Z H, Sigal L. Learning to generate posters of scientific papers. In Proc. the 30th AAAI Conference on Artificial Intelligence, February 2016, pp.51-57.Google Scholar
  14. [14]
    Holland J H. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. A Bradford Book, 1992.Google Scholar
  15. [15]
    Goldberg D E. Genetic Algorithms in Search, Optimization and Machine Learning (1st edition). Addison Wesley, 1989.Google Scholar
  16. [16]
    Gajos K, Weld D S. Preference elicitation for interface optimization. In Proc. the 18th Annual ACM Symposium on User Interface Software and Technology, October 2005, pp.173-182.Google Scholar
  17. [17]
    Sarrafzadeh M, Lee D T. Algorithmic Aspects of VLSI Layout. World Scientific Pub. Co. Inc., 1993.Google Scholar
  18. [18]
    Battista G D, Eades P, Tamassia R, Tollis I G. Graph Drawing: Algorithms for the Visualization of Graphs (1st edition). Pearson, 1998.Google Scholar
  19. [19]
    Matsui Y, Yamasaki T, Aizawa K. Interactive manga retargeting. In Proc. ACM SIGGRAPH 2011 Posters, August 2011, Article No. 35.Google Scholar
  20. [20]
    Hoashi K, Ono C, Ishii D, Watanabe H. Automatic preview generation of comic episodes for digitized comic search. In Proc. the 19th ACM International Conference on Multimedia, November 2011, pp.1489-1492.Google Scholar
  21. [21]
    Qu Y, Pang W M, Wong T T, Heng P N. Richnesspreserving manga screening. ACM Transactions on Graphics, 2008, 27(5): Article No. 155.Google Scholar
  22. [22]
    Arai K, Herman T. Method for automatic e-comic scene frame extraction for reading comic on mobile devices. In Proc. the 7th International Conference on Information Technology: New Generations, April 2010, pp.370-375.Google Scholar
  23. [23]
    Pang X, Cao Y, Lau R W H, Chan A B. A robust panel extraction method for manga. In Proc. the 22nd ACM International Conference on Multimedia, November 2014, pp.1125-1128.Google Scholar
  24. [24]
    Jing G, Hu Y, Guo Y, Yu Y, Wang W. Content-aware video2comics with manga-style layout. IEEE Transactions on Multimedia, 2015, 17(12): 2122-2133.CrossRefGoogle Scholar
  25. [25]
    Cao Y, Chan A B, Lau R W H. Automatic stylistic manga layout. ACM Transactions on Graphics, 2012, 31(6): Article No. 141.Google Scholar
  26. [26]
    Lodi A, Martello S, Monaci M. Two-dimensional packing problems: A survey. European Journal of Operational Research, 2002, 141(2): 241-252.MathSciNetCrossRefzbMATHGoogle Scholar
  27. [27]
    Stockmeyer L. Optimal orientations of cells in slicing floorplan designs. Information and Control, 1983, 57(2/3): 91-101.MathSciNetCrossRefzbMATHGoogle Scholar
  28. [28]
    Kaser O, Lemire D. Tag-cloud drawing: Algorithms for cloud visualization. arXiv:0703109, 2007., August 2018.
  29. [29]
    Lok S, Feiner S. A survey of automated layout techniques for information presentations. In Proc. the 1st International Symposium on Smart Graphics, March 2001, pp.61-68.Google Scholar
  30. [30]
    Jacobs C, Li W, Schrier E, Bargeron D, Salesin D. Adaptive grid-based document layout. ACM Transactions on Graphics, 2003, 22(3): 838-847.CrossRefGoogle Scholar
  31. [31]
    Harrington S J, Naveda J F, Jones R P, Roetling P, Thakkar N. Aesthetic measures for automated document layout. In Proc. the 2004 ACM Symposium on Document Engineering, October 2004, pp.109-111.Google Scholar
  32. [32]
    Purvis L, Harrington S, O’Sullivan B, Freuder E C. Creating personalized documents: An optimization approach. In Proc. the 2003 ACM Symposium on Document Engineering, Nov. 2003, pp.68-77.Google Scholar
  33. [33]
    Pinto A, Pedrini H, Schwartz W R, Rocha A. Face spoofing detection through visual codebooks of spectral temporal cubes. IEEE Transactions on Image Processing, 2015, 24(12): 4726-4740.MathSciNetCrossRefGoogle Scholar
  34. [34]
    Murphy K. The bayes net toolbox for Matlab. Technical Report, Computing Science and Statistics, 2001,, Nov. 2018.
  35. [35]
    Fung R, Chang K C. Weighing and integrating evidence for stochastic simulation in Bayesian networks. Machine Intelligence and Pattern Recognition, 1990, 10: 209-220.CrossRefGoogle Scholar
  36. [36]
    Zhao Y, Zhu S C. Image parsing with stochastic scene grammar. In Proc. the 25th International Conference on Neural Information Processing Systems, December 2011, pp.73-81.Google Scholar
  37. [37]
    Chen Z, Mukherjee A, Liu B, Hsu M, Castellanos M, Ghosh R. Leveraging multi-domain prior knowledge in topic models. In Proc. the 23rd International Joint Conference on Artificial Intelligence, August 2013, pp.2071-2077.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Yu-Ting Qiang
    • 1
  • Yan-Wei Fu
    • 2
  • Xiao Yu
    • 1
  • Yan-Wen Guo
    • 1
    Email author
  • Zhi-Hua Zhou
    • 1
  • Leonid Sigal
    • 3
  1. 1.National Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina
  2. 2.School of Data ScienceFudan UniversityShanghaiChina
  3. 3.Disney Research PittsburghPittsburghUSA

Personalised recommendations