Pattern understanding and synthesis based on layout tree descriptor

  • Xinwei Zhang
  • Jin WangEmail author
  • Guodong Lu
  • Xusheng Zhang
Original Article


Synthesis from existing examples is a promising way to generate new patterns. However, pattern synthesis is challenging because it is difficult to understand and generate complex structures in patterns. In this paper, we propose an approach based on the layout tree descriptor (LTD) to understand and synthesize patterns from existing ones. The LTD is a binary tree that parametrically describes all primitives, layouts, their dependencies and hierarchies in a pattern. The LTD can be constructed automatically with proposed instance grouping, layout recognition, hyper-primitive matching and tree merging algorithms to realize pattern understanding. To meet specialists’ requirements for detailed modification and recombination of patterns, we designed LTD operations including add, remove, replace and grafting operations to allow users to get new patterns by simply adjusting the LTDs. For stylized synthesis, we gave the computing method of LTD similarity. Therefore, the styles of results and input can be compared and users can control generated serialized results by setting the input pattern weights. To meet user’s implicit preferences and provide novelty in creative design, we propose an evolutionary approach to creative synthesis. The system generates new patterns continuously based on LTD grafting, meanwhile user selection of preferred patterns will guide the direction of evolution. Experiments using the developed prototype system show that our approach can synthesize novel and complex patterns effectively, meeting different requirements in practice and providing plenty of digital textures for products.


Patterns Layouts Synthesis Design tools Graphical models Design space exploration 



We wish to acknowledge Dr. Cui of Shandong Normal University and Prof. Tang of Shaanxi Fashion Engineering University for their permission of using embroidery patterns extracted from their paper in this study.


This study is funded by the National Natural Science Foundation of China (51775492) and Robotics Institute of Zhejiang University (K18-508116-001).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. 1.
    Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems (NIPS), Montreal, Canada, 2014, pp. 2672–2680 (2014) Google Scholar
  2. 2.
    Nguyen, A., Clune, J., Bengio, Y., Dosovitskiy, A., Yosinski, J.: Plug & play generative networks: conditional iterative generation of images in latent space. In: Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, USA, 2017, pp. 3510–3520 (2017)Google Scholar
  3. 3.
    Reinert, B., Ritschel, T., Seidel, H.: Interactive by-example design of artistic packing layouts. ACM Trans. Graph. 32, 1–7 (2013)Google Scholar
  4. 4.
    Xu, P., Fu, H., Tai, C.L., Igarashi, T.: GACA: group-aware command-based arrangement of graphic elements, In: ACM Conference on Human Factors in Computing Systems (CHI), Seoul, Republic of Korea, 2015, pp. 2787–2795 (2015)Google Scholar
  5. 5.
    Fish, N., Averkiou, M., Kaick, O.V., Sorkine-Hornung, O., Cohen-Or, D., Mitra, N.J.: Meta-representation of shape families. ACM Trans. Graph. 33(4), 1–11 (2014)zbMATHGoogle Scholar
  6. 6.
    O’Donovan, P., Agarwala, A., Hertzmann, A.: Learning layouts for single-pagegraphic designs. IEEE Trans. Vis. Comput. Graph. 20(8), 1200–1213 (2014)Google Scholar
  7. 7.
    Lee, A.J.T., Chiu, H.P.: 2D Z-string: a new spatial knowledge representation for image databases. Pattern Recognit. Lett. 24(16), 3015–3026 (2003)Google Scholar
  8. 8.
    Matsakis, P., Wendling, L.: A new way to represent the relative position between areal objects. IEEE Trans. Pattern Anal. Mach. Intell. 21(7), 634–643 (1999)Google Scholar
  9. 9.
    Matsakis, P., Keller, J.M., Wendling, L., Marjamaa, J., Sjahputera, O.: Linguistic description of relative positions in images. IEEE Trans. Syst. Man Cybern. Part B (Cybern.). 31(4), 573–588 (2001).
  10. 10.
    Matsakis, P., Wawrzyniak, L., Ni, J.: Relative positions in words: a system that builds descriptions around Allen relations. Int. J. Geogr. Inf. Sci. 24(1), 1–23 (2010)Google Scholar
  11. 11.
    Hiller, B.: Space is the machine: a configurational theory of architecture. J. Urban Des. 3, 288–313 (2007)Google Scholar
  12. 12.
    Lee, J.H., Ostwald, M.J., Gu, N.: A syntactical and grammatical approach to architectural configuration, analysis and generation. Archit. Sci. Rev. 58(3), 189–204 (2015)Google Scholar
  13. 13.
    Fisher, M., Savva, M., Hanrahan, P.: Characterizing structural relationships in scenes using graph kernels. ACM Trans. Graph. 30(4), 1–12 (2011)Google Scholar
  14. 14.
    Ślusarczyk, G.: Graph-based representation of design properties in creating building floorplans. Comput. Aided Des. 95, 24–39 (2017)MathSciNetGoogle Scholar
  15. 15.
    Jaiswal, P., Huang, J., Rai, R.: Assembly-based conceptual 3D modeling with unlabeled components using probabilistic factor graph. Comput. Aided Des. 74, 45–54 (2016)Google Scholar
  16. 16.
    Nan, L., Sharf, A., Xie, K., Wong, T.T., Deussen, O., Cohenor, D., Chen, B.: Conjoining Gestalt rules for abstraction of architectural drawings. J. Comput. Aided Des. Comput. Graph. 30(6), 1–10 (2012)Google Scholar
  17. 17.
    Nelson, G.: Juno, a constraint-based graphics system. In: Conference on Computer Graphics and Interactive Techniques, New York, NY, USA, 1985, pp. 235–243 (1985)Google Scholar
  18. 18.
    Ryall, K., Marks, J., Shieber, S.M.: An interactive system for drawing graphs. In: Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology, Banff, Alberta, Canada, 1997, pp. 387–394 (1997)Google Scholar
  19. 19.
    Baudisch, P., Cutrell, E., Hinckley, K., Eversole, A.: Snap-and-go: helping users align objects without the modality of traditional snapping. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI), Portland, Oregon, USA, 2005, pp. 301–310 (2005)Google Scholar
  20. 20.
    O’Donovan, P., Agarwala, A., Hertzmann, A.: DesignScape: design with interactive layout suggestions. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI), Seoul, Republic of Korea, 2015, pp. 1221–1224 (2015)Google Scholar
  21. 21.
    Guerrero, P., Bernstein, G., Li, W., Mitra, N.: PATEX: exploring pattern variations. ACM Trans. Graph. 35(4), 1–13 (2016)Google Scholar
  22. 22.
    Jesus, D., Coelho, A., Sousa, A.A.: Layered shape grammars for procedural modelling of buildings. Vis. Comput. 32(6), 933–943 (2016)Google Scholar
  23. 23.
    Jowers, I., Earl, C., Stiny, G.: Shape computations without compositions. Commun. Comput. Inf. Sci. 724, 348–365 (2017)Google Scholar
  24. 24.
    Bohl, E., Terraz, O., Ghazanfarpour, D.: Modeling fruits and their internal structure using parametric 3Gmap L-systems. Vis. Comput. 31(6), 1–11 (2015)Google Scholar
  25. 25.
    Cui, J., Tang, M.X.: Integrating shape grammars into a generative system for Zhuang ethnic embroidery design exploration. Comput. Aided Des. 45(3), 591–604 (2013)MathSciNetGoogle Scholar
  26. 26.
    Ruiz-Montiel, M., Belmonte, M.V., Boned, J., Mandow, L., Millán, E., Badillo, A., Pérez-De-La-Cruz, J.L.: Layered shape grammars. Comput. Aided Des. 56, 104–119 (2014)Google Scholar
  27. 27.
    Kielarova, S.W., Pradujphongphet, P., Bohez, E.L.J.: New interactive-generative design system: hybrid of shape grammar and evolutionary design—an application of jewelry design. In: Advances in Swarm and Computational Intelligence, Beijing, China, 2015, pp. 302–313 (2015)Google Scholar
  28. 28.
    Dino, I.G.: An evolutionary approach for 3D architectural space layout design exploration. Autom. Constr. 69, 131–150 (2016)Google Scholar
  29. 29.
    Lu, S., Mok, P.Y., Jin, X.: From design methodology to evolutionary design: an interactive creation of marble-like textile patterns. Eng. Appl. Artif. Intell. 32, 124–135 (2014)Google Scholar
  30. 30.
    Guo, X., Lin, J., Xu, K., Jin, X.: Creature grammar for creative modeling of 3D monsters. Graph. Models 75(5), 376–389 (2014)Google Scholar
  31. 31.
    Zhou, J., Chen, X.: Convertible furniture design. Comput. Graph. 70, 165–175 (2017)Google Scholar
  32. 32.
    Alhashim, I., Li, H., Xu, K., Cao, J., Ma, R., Zhang, H.: Topology-varying 3D shape creation via structural blending. ACM Trans. Graph. 33(4), 1–10 (2014)Google Scholar
  33. 33.
    Yeh, Y., Měch, R.: Detecting symmetries and curvilinear arrangements in vector art. Comput. Graph. Forum 28(2), 707–716 (2010)Google Scholar
  34. 34.
    Yeh, Y., Breeden, K., Yang, L., Fisher, M., Hanrahan, P.: Synthesis of tiled patterns using factor graphs. ACM Trans. Graph. 22(1), 1–13 (2013)zbMATHGoogle Scholar
  35. 35.
    Talton, J.O., Gibson, D., Yang, L., Hanrahan, P., Koltun, V.: Exploratory modeling with collaborative design spaces. ACM Trans. Graph. 28(5), 1–10 (2009)Google Scholar
  36. 36.
    Kusiak, A., Heragu, S.S.: The facility layout problem. Eur. J. Oper. Res. 29(3), 229–251 (1987)MathSciNetzbMATHGoogle Scholar
  37. 37.
    Vitayasak, S., Pongcharoen, P., Hicks, C.: A tool for solving stochastic dynamic facility layout problems with stochastic demand using either a Genetic Algorithm or modified Backtracking Search Algorithm. Int. J. Prod. Econ. 190, 146–157 (2017)Google Scholar
  38. 38.
    Xu, P., Fu, H., Igarashi, T., Tai, C.: Global beautification of layouts with interactive ambiguity resolution, In: Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology, Honolulu, Hawaii, USA, 2014, pp. 243–252 (2014)Google Scholar
  39. 39.
    Hua, H.: Irregular architectural layout synthesis with graphical inputs. Autom. Constr. 72, 388–396 (2016)Google Scholar
  40. 40.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)Google Scholar
  41. 41.
    Valor, M., Albert, F., Gomis, J.M., Contero, M.: Analysis tool for cataloguing textile and tile pattern designs. Lect. Notes Comput. Sci. 2669, 569–578 (2003)Google Scholar
  42. 42.
    Huang, H., Zhang, Y., Yu, X.: Study and implementation of ceramic pattern intelligent design. In: International Conference on Computing, Control and Industrial Engineering (CCIE), Wuhan, China, 2010, pp. 132–135 (2010)Google Scholar
  43. 43.
    Ge, Y.: Compositional rules for graphic system of decorative pattern art based on digital image. Paper Asia 1(8), 52–55 (2018)Google Scholar
  44. 44.
    Yao, J., Yu, H., Hu, R.: A new sparse representation-based object segmentation framework. Vis. Comput. 33(2), 179–192 (2017)Google Scholar
  45. 45.
    Qi, J., Xin, F., Yu, L., Haojie, L., Zhongxuan, L., He, G.: Hierarchical projective invariant contexts for shape recognition. Pattern Recognit. 52, 358–374 (2016)Google Scholar
  46. 46.
    Wei, S., Yuan, J., Gao, W., Dan, Z., Wang, X.: Shape recognition by bag of skeleton-associated contour parts. Pattern Recognit. Lett. 83, 321–329 (2016)Google Scholar
  47. 47.

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.State Key Laboratory of Fluid Power and Mechatronic SystemsZhejiang UniversityHangzhouPeople’s Republic of China
  2. 2.College of Computer Science and TechnologyZhejiang UniversityHangzhouPeople’s Republic of China

Personalised recommendations