Building hierarchical structures for 3D scenes with repeated elements

  • Xi Zhao
  • Zhenqiang Su
  • Taku Komura
  • Xinyu YangEmail author
Original Article


We propose a novel hierarchy construction algorithm for 3D scenes with repeated elements, such as classrooms with multiple desk–chair pairs. Most existing algorithms focus on scenes such as bedrooms or living rooms, which rarely contain repeated patterns. Consequently, such methods may not recognize repeated patterns, which are vital for understanding the structure and context of scenes such as classrooms. Therefore, we propose a new global optimization algorithm for recognizing repeated patterns and building hierarchical structures based on repeated patterns. First, we find a repeated template by calculating the coverage ratios and frequencies of many substructures in a scene. Once the repeated template has been determined, a minimum cost maximum flow problem can be solved to find all instances (repetitions) of it in the scene and then group objects accordingly. Second, we group objects in the region outside the repeated elements according to their adjacency. Finally, based on these two sets of results, we build the hierarchy of the entire scene. We test this hierarchy construction algorithm on the Princeton and SceneNN databases and show that our algorithm can correctly find repeated patterns and construct a hierarchy that is more similar to the ground truth than the results of previous methods.


3D scene Scene analysis Hierarchy Repeated patterns 



This study was funded by the National Natural Science Foundation of China (61602366) and the China Postdoctoral Science Foundation (2015M582664).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Supplementary material


  1. 1.
    Ahuja, N., Todorovic, S.: Extracting texels in 2.1D natural textures. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar
  2. 2.
    Alhashim, I., Li, H., Xu, K., Cao, J., Ma, R., Zhang, H.: Topology-varying 3D shape creation via structural blending. ACM Trans. Gr. (TOG) 33(4), 158 (2014)Google Scholar
  3. 3.
    Bokeloh, M., Wand, M., Seidel, H.P.: A connection between partial symmetry and inverse procedural modeling. ACM Trans. Gr. (TOG) 29(4), 104 (2010)Google Scholar
  4. 4.
    Cheng, M.M., Zhang, F.L., Mitra, N.J., Huang, X., Hu, S.M.: RepFinder: finding approximately repeated scene elements for image editing. ACM Trans. Gr. 29(4), 1 (2010)CrossRefGoogle Scholar
  5. 5.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 3rd edn. The MIT Press, Cambridge (2009)zbMATHGoogle Scholar
  6. 6.
    Fisher, M., Savva, M., Hanrahan, P.: Characterizing structural relationships in scenes using graph kernels. ACM Trans. Gr. (TOG) 30, 34 (2011)Google Scholar
  7. 7.
    Gal, R., Cohen-Or, D.: Salient geometric features for partial shape matching and similarity. ACM Trans. Gr. (TOG) 25(1), 130–150 (2006)CrossRefGoogle Scholar
  8. 8.
    Golovinskiy, A., Funkhouser, T.: Consistent segmentation of 3D models. Comput. Gr. 33(3), 262–269 (2009)CrossRefGoogle Scholar
  9. 9.
    Hu, R., Zhu, C., van Kaick, O., Liu, L., Shamir, A., Zhang, H.: Interaction context (ICON): towards a geometric functionality descriptor. ACM Trans. Gr. 34, 83:1–83:12 (2015)CrossRefGoogle Scholar
  10. 10.
    Huang, H., Zhang, L., Zhang, H.C.: RepSnapping: efficient image cutout for repeated scene elements. Comput. Gr. Forum 30(7), 2059–2066 (2011)CrossRefGoogle Scholar
  11. 11.
    Hueting, M., Monszpart, A., Mellado, N.: MCGraph: multi-criterion representation for scene understanding. In: SIGGRAPH Asia 2014 Indoor Scene Understanding Where Graphics Meets Vision, p. 3. ACM (2014)Google Scholar
  12. 12.
    Kalogerakis, E., Chaudhuri, S., Koller, D., Koltun, V.: A probabilistic model for component-based shape synthesis. ACM Trans. Gr. (TOG) 31(4), 55 (2012)Google Scholar
  13. 13.
    Li, J., Xu, K., Chaudhuri, S., Yumer, E., Zhang, H., Guibas, L.: GRASS: generative recursive autoencoders for shape structures. ACM Trans. Gr. 36(4), 1–12 (2017). arXiv:1705.02090 CrossRefGoogle Scholar
  14. 14.
    Liu, Y., Collins, R., Tsin, Y.: A computational model for periodic pattern perception based on frieze and wallpaper groups. IEEE Trans. Pattern Anal. Mach. Intell. 26(3), 354–371 (2004)CrossRefGoogle Scholar
  15. 15.
    Liu, S., Martin, R.R., Langbein, F.C., Rosin, P.L.: Segmenting periodic reliefs on triangle meshes. In: Martin, R., Sabin, M., Winkler, J. (eds.) Mathematics of Surfaces XII, vol. 4647, pp. 290–306. Springer, Berlin (2007)CrossRefGoogle Scholar
  16. 16.
    Liu, T., Chaudhuri, S., Kim, V.G., Huang, Q., Mitra, N.J., Funkhouser, T.: Creating consistent scene graphs using a probabilistic grammar. ACM Trans. Gr. 33(6), 211:1–211:12 (2014)zbMATHGoogle Scholar
  17. 17.
    MacDonald, D., Lang, J., McAllister, M.: Evaluation of colour image segmentation hierarchies. In: The 3rd Canadian Conference on Computer and Robot Vision, 2006, p. 27 (2006)Google Scholar
  18. 18.
    Mitra, N., Wand, M., Zhang, H.R., Cohen-Or, D., Kim, V., Huang, Q.X.: Structure-aware shape processing. In: SIGGRAPH Asia 2013 Courses, SA ’13, pp. 1:1–1:20. ACM, New York (2013)Google Scholar
  19. 19.
    Moulin, M., Dutré, P.: On the use of local ray termination for efficiently constructing qualitative BSPs, BIHs and (S)BVHs. Vis. Comput. (2018).
  20. 20.
    Nguyen, D.T., Hua, B.S., Yu, L.F., Yeung, S.K.: A Robust 3D–2D Interactive Tool for Scene Segmentation and Annotation (2016). arXiv:1610.05883 [cs]. arXiv:1610.05883
  21. 21.
    Paraboschi, L., Biasotti, S., Falcidieno, B.: 3D scene comparison using topological graphs. In: Eurographics Italian Chapter Conference, pp. 87–93 (2007)Google Scholar
  22. 22.
    Sappa, A.D., Garcia, M.A.: Generating compact representations of static scenes by means of 3D object hierarchies. Vis. Comput. 23(2), 143–154 (2007)CrossRefGoogle Scholar
  23. 23.
    Sidi, O., van Kaick, O., Kleiman, Y., Zhang, H., Cohen-Or, D.: Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. ACM Trans. Gr. (TOG) 30, 126 (2011)Google Scholar
  24. 24.
    Wang, Y., Xu, K., Li, J., Zhang, H., Shamir, A., Liu, L., Cheng, Z., Xiong, Y.: Symmetry hierarchy of man-made objects. In: Computer Graphics Forum, vol. 30, pp. 287–296. Wiley Online Library (2011)Google Scholar
  25. 25.
    Xu, K., Ma, R., Zhang, H., Zhu, C., Shamir, A., Cohen-Or, D., Huang, H.: Organizing heterogeneous scene collections through contextual focal points. ACM Trans. Gr. (TOG) 33(4), 35 (2014)Google Scholar
  26. 26.
    Zhang, F.-L., Cheng, M.-M., Jia, J., Shi-Min, H.: ImageAdmixture: putting together dissimilar objects from groups. IEEE Trans. Vis. Comput. Gr. 18(11), 1849–1857 (2012)CrossRefGoogle Scholar
  27. 27.
    Zhao, X., Hu, R., Guerrero, P., Mitra, N., Komura, T.: Relationship templates for creating scene variations. ACM Trans. Gr. 35(6), 207:1–207:13 (2016)Google Scholar
  28. 28.
    Zhao, X., Wang, H., Komura, T.: Indexing 3D scenes using the interaction bisector surface. ACM Trans. Gr. (TOG) 33(3), 22:1–22:14 (2014)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Electronic and Information EngineeringXi’an Jiaotong UniversityXi’anChina
  2. 2.School of Software EngineeringXi’an Jiaotong UniversityXi’anChina
  3. 3.School of InformaticsEdinburgh UniversityEdinburghUK

Personalised recommendations