Advertisement

The Visual Computer

, Volume 32, Issue 2, pp 243–256 | Cite as

Spatio-temporal segmentation for the similarity measurement of deforming meshes

  • Guoliang LuoEmail author
  • Frederic Cordier
  • Hyewon Seo
Original Article

Abstract

Although there have been a large body of works on computing the similarity of static shapes, similarity judgments on deforming meshes are not studied well. In this study, we investigate a similarity measurement method for comparing two deforming meshes. Based on the degree of deformation, we first binarily label each triangle within each frame as either ‘deformed’ or ‘rigid’, then merge the ‘deformed’ triangles in both spatial and temporal domains for the segmentation. The segmentation results are encoded in a form of evolving graph, with an aim of obtaining a compact representation of the motion of the mesh. Finally, we formulate the similarity measurement as a sequence matching problem: after clustering similar graphs and assigning each of the graphs with the cluster labels, each deforming mesh is represented with a sequence of labels. Then, we apply a sequence alignment algorithm to compute the locally optimal alignment between the two label sequences, and to compute the similarity by normalizing the alignment score. The experimental results over several datasets show that the similarities of animation data can be captured correctly using our approach. This may be significant, as it solves a problem that cannot be handled by current approaches.

Keywords

Spatio-temporal segmentation Deforming mesh Similarity Sequence alignment 

Notes

Acknowledgments

We first would like to thank Professor Adrian Hilton and Dr. Peng Huang from the University of Surrey, for providing us a set of synthetic deforming meshes, which have been used for evaluation. We are grateful to MIRALab at the University of Geneva, who has provided us with the scanned face data. This has allowed us to make the highly resoluted facial animations “Face_2” and “Face_3” by transferring our mocap facial expressions to the scanned meshes. We would like to thank Frédéric Larue and Olivier Génevaux for providing us with the facial motion capture data, and Arash Habibi for providing us the “Horse_2” data. We also would like to thank Vasyl Mykhalchuk, who has developed a tool for visualizing the evolving graphs. This work has been jointly supported by the French national project SHARED (Shape Analysis and Registration of People Using Dynamic Data, No. 10-CHEX-014-01), and the Science and Technology Research Project of Jiangxi Provincial Department of Education (GJJ14246).

Supplementary material

Supplementary material 1 (mov 7966 KB)

References

  1. 1.
    Autodesk Inc.: 3D Studio Max. http://www.autodesk.com/products/3ds-max/overview. Accessed 23 Nov 2015
  2. 2.
    Sumner, R., Popovic, J.: Mesh data from deformation transfer for triangle meshes. Computer Graphics Group at MIT. http://people.csail.mit.edu/sumner/research/deftransfer/data.html. Accessed 23 Nov 2015
  3. 3.
    Bronstein, A.M., Bronstein, M.M., Castellani, U., Falcidieno, B., Fusiello, A., Godil, A., Guibas, L.J., Kokkinos, I., Lian, Z., Ovsjanikov, M., Patane, G., Spagnuolo, M., Toldo, R.: Project: tools for non-rigid shape comparison and analysis. http://tosca.cs.technion.ac.il/. Accessed 23 Nov 2015
  4. 4.
    Ankerst, M., Kastenmüller, G., Kriegel, H.-P., Seidl, T.: 3d shape histograms for similarity search and classification in spatial databases. In: Advances in Spatial Databases, pp. 207–226. Springer (1999)Google Scholar
  5. 5.
    Attene, M., Falcidieno, B., Spagnuolo, M.: Hierarchical mesh segmentation based on fitting primitives. Vis. Comput. 22(3), 181–193 (2006)CrossRefGoogle Scholar
  6. 6.
    Attene, M., Katz, S., Mortara, M., Patané, G., Spagnuolo, M., Tal, A.: Mesh segmentation—a comparative study. In: IEEE International Conference on Shape Modeling and Applications, 2006. SMI (2006)Google Scholar
  7. 7.
    Barton, G.J.: An efficient algorithm to locate all locally optimal alignments between two sequences allowing for gaps. Comput. Appl. Biosci.: CABIOS 9(6), 729–734 (1993)Google Scholar
  8. 8.
    Benhabiles, H., Lavoué, G., Vandeborre, J.-P., Daoudi, M.: Learning boundary edges for 3d-mesh segmentation. In: Computer Graphics Forum, vol. 30, pp. 2170–2182. Wiley Online Library (2011)Google Scholar
  9. 9.
    Budd, C., Huang, P., Klaudiny, M., Hilton, A.: Global non-rigid alignment of surface sequences. Int. J. Comput. Vis. 102(1–3), 256–270 (2013)CrossRefMathSciNetGoogle Scholar
  10. 10.
    Chen, X., Golovinskiy, A., Funkhouser, T.: A benchmark for 3d mesh segmentation. In: ACM Transactions on Graphics (TOG), vol. 28, p. 73. ACM (2009)Google Scholar
  11. 11.
    Crandall, S.H., Lardner, T.J., Archer, R.R., Cook, N.H., Dahl, N.C.: An introduction to the mechanics of solidsGoogle Scholar
  12. 12.
    Cyr, C.M., Kimia, B.B.: 3d object recognition using shape similarity-based aspect graph. In: Eighth IEEE International Conference on Computer Vision, 2001. ICCV 2001. Proceedings, vol. 1, pp. 254–261. IEEE (2001)Google Scholar
  13. 13.
    Gkalelis, N., Kim, H., Hilton, A., Nikolaidis, N., Pitas, I.: The i3dpost multi-view and 3d human action/interaction database. In: Conference for Visual Media Production, 2009. CVMP’09, pp. 159–168. IEEE (2009)Google Scholar
  14. 14.
    Golovinskiy, A., Funkhouser, T.: Consistent segmentation of 3d models. Comput. Gr. 33(3), 262–269 (2009)CrossRefGoogle Scholar
  15. 15.
    Gusfield, D.: Algorithms on Strings, Trees and Sequences: Computer Science and Computational Biology. Cambridge University Press, Cambridge (1997)CrossRefzbMATHGoogle Scholar
  16. 16.
    Huang, Q., Koltun, V., Guibas, L.: Joint shape segmentation with linear programming. In: ACM Transactions on Graphics (TOG), vol. 30, p. 125. ACM (2011)Google Scholar
  17. 17.
    Inaba, M., Katoh, N., Imai, H.: Applications of weighted voronoi diagrams and randomization to variance-based k-clustering. In: Proceedings of the Tenth Annual Symposium on Computational Geometry, pp. 332–339. ACM (1994)Google Scholar
  18. 18.
    Johnson, A.E.: Spin-images: a representation for 3-D surface matching. Ph.D. thesis, Citeseer (1997)Google Scholar
  19. 19.
    Jolliffe, I.: Principal Component Analysis. Wiley Online Library, New York (2005)CrossRefGoogle Scholar
  20. 20.
    Kalafatlar, E., Yemez, Y.: 3d articulated shape segmentation using motion information. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 3595–3598. IEEE (2010)Google Scholar
  21. 21.
    Kalogerakis, E., Hertzmann, A., Singh, K.: Learning 3d mesh segmentation and labeling. ACM Trans. Gr. (TOG) 29(4), 102 (2010)Google Scholar
  22. 22.
    Katz, S., Leifman, G., Tal, A.: Mesh segmentation using feature point and core extraction. Vis. Comput. 21(8–10), 649–658 (2005)CrossRefGoogle Scholar
  23. 23.
    Kazhdan, M., Funkhouser, T., Rusinkiewicz, S.: Rotation invariant spherical harmonic representation of 3d shape descriptors. In: Symposium on Geometry Processing, vol. 6 (2003)Google Scholar
  24. 24.
    Kruskall, J.B., Liberman, M.: The symmetric time warping algorithm: from continuous to discrete. In: The Theory and Practice of Sequence Comparison, in time Warps, String Edits and Macromolecules (1983)Google Scholar
  25. 25.
    Lee, T.-Y., Wang, Y.-S., Chen, T.-G.: Segmenting a deforming mesh into near-rigid components. Vis. Comput. 22(9–11), 729–739 (2006)CrossRefGoogle Scholar
  26. 26.
    Luo, G., Cordier, F., Seo, H.: Similarity of deforming meshes based on spatio-temporal segmentation. In: 3D Object Retrieval Workshop, 3DOR (2014)Google Scholar
  27. 27.
    Mamou, K., Zaharia, T., Prêteux, F.: A skinning approach for dynamic 3d mesh compression. Comput. Anim. Virtual Worlds 17(3–4), 337–346 (2006)CrossRefGoogle Scholar
  28. 28.
    Müller, M., Röder, T., Clausen, M.: Efficient content-based retrieval of motion capture data. In: ACM Transactions on Graphics (TOG), vol. 24, pp. 677–685. ACM (2005)Google Scholar
  29. 29.
    Neuhaus, M., Riesen, K., Bunke, H.: Fast suboptimal algorithms for the computation of graph edit distance. In: Structural, Syntactic, and Statistical Pattern Recognition, pp. 163–172. Springer (2006)Google Scholar
  30. 30.
    Riesen, K., Bunke, H.: Graph classification based on vector space embedding. Int. J. Pattern Recognit. Artif. Intell. 23(06), 1053–1081 (2009)CrossRefGoogle Scholar
  31. 31.
    Riesen, K., Bunke, H.: Reducing the dimensionality of dissimilarity space embedding graph kernels. Eng. Appl. Artif. Intell. 22(1), 48–56 (2009)CrossRefGoogle Scholar
  32. 32.
    Sattler, M., Sarlette, R., Klein, R.: Simple and efficient compression of animation sequences. In: Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 209–217. ACM (2005)Google Scholar
  33. 33.
    Shamir, A.: A survey on mesh segmentation techniques. In: Computer Graphics Forum, vol. 27, pp. 1539–1556. Wiley Online Library (2008)Google Scholar
  34. 34.
    Shapira, L., Shamir, A., Cohen-Or, D.: Consistent mesh partitioning and skeletonisation using the shape diameter function. Vis. Comput. 24(4), 249–259 (2008)CrossRefGoogle Scholar
  35. 35.
    Sidi, O., van Kaick, O., Kleiman, Y., Zhang, H., Cohen-Or, D.: Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. In: ACM Transactions on Graphics (TOG), vol. 30, p. 126. ACM (2011)Google Scholar
  36. 36.
    Smith, T.F., Waterman, M.S.: Identification of common molecular subsequences. J. Mol. Biol. 147(1), 195–197 (1981)CrossRefGoogle Scholar
  37. 37.
    Starck, J., Hilton, A.: Surface capture for performance-based animation. Comput. Gr. Appl. IEEE 27(3), 21–31 (2007)CrossRefGoogle Scholar
  38. 38.
    Stigler, S.M.: Francis Galton’s account of the invention of correlation. Stat. Sci. 4(2):73–79 (1989)Google Scholar
  39. 39.
    Sumner, R.W., Zwicker, M., Gotsman, C., Popović, J.: Mesh-based inverse kinematics. In: ACM Transactions on Graphics (TOG), vol. 24, pp. 488–495. ACM (2005)Google Scholar
  40. 40.
    Sundar, H., Silver, D., Gagvani, N., Dickinson, S.: Skeleton based shape matching and retrieval. In: Shape Modeling International, 2003, pp. 130–139. IEEE (2003)Google Scholar
  41. 41.
    Wang, Y., Gong, M., Wang, T., Cohen-Or, D., Zhang, H., Chen, B.: Projective analysis for 3d shape segmentation. ACM Trans. Gr. (TOG) 32(6), 192 (2013)Google Scholar
  42. 42.
    Wu, S., Xia, S., Wang, Z., Li, C.: Efficient motion data indexing and retrieval with local similarity measure of motion strings. Vis. Comput. 25(5–7), 499–508 (2009)CrossRefGoogle Scholar
  43. 43.
    Wuhrer, S., Brunton, A.: Segmenting animated objects into near-rigid components. Vis. Comput. 26(2), 147–155 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Jiangxi Normal University (MIMLab)NanchangChina
  2. 2.LMIAUniversité de Haute Alsace (LMIA, EA 3993)MulhouseFrance
  3. 3.Université de Strasbourg (ICube, UMR 7357, CNRS)StrasbourgFrance

Personalised recommendations