Efficient Single-View 3D Co-segmentation Using Shape Similarity and Spatial Part Relations

  • Nikita AraslanovEmail author
  • Seongyong Koo
  • Juergen Gall
  • Sven Behnke
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9796)


The practical use of the latest methods for supervised 3D shape co-segmentation is limited by the requirement of diverse training data and a watertight mesh representation. Driven by practical considerations, we assume only one reference shape to be available and the query shape to be provided as a partially visible point cloud. We propose a novel co-segmentation approach that constructs a part-based object representation comprised of shape appearance models of individual parts and isometric spatial relations between the parts. The partial query shape is pre-segmented using planar cuts, and the segments accompanied by the learned representation induce a compact Conditional Random Field (CRF). CRF inference is performed efficiently by \(A^*\)-search with global optimality guarantees. A comparative evaluation with two baselines on partial views generated from the Labelled Princeton Segmentation Benchmark and point clouds recorded with an RGB-D sensor demonstrate superiority of the proposed approach both in accuracy and efficiency.


Point Cloud Conditional Random Field Move Little Square Shape Part Reference Shape 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported by German Research Foundation (DFG) under grant BE 2556/12 ALROMA in priority programme SPP 1527 Autonomous Learning.


  1. 1.
    Alhashim, I., Xu, K., Zhuang, Y., Cao, J., Simari, P., Zhang, H.: Deformation-driven topology-varying 3D shape correspondence. TOG 34(6), 236 (2015)CrossRefGoogle Scholar
  2. 2.
    Bergtholdt, M., Kappes, J., Schmidt, S., Schnörr, C.: A study of parts-based object class detection using complete graphs. IJCV 87(1–2), 93–117 (2010)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bronstein, A.M., Bronstein, M.M., Kimmel, R., Mahmoudi, M., Sapiro, G.: A Gromov-Hausdorff framework with diffusion geometry for topologically-robust non-rigid shape matching. IJCV 89(2–3), 266–286 (2010)CrossRefGoogle Scholar
  4. 4.
    Bronstein, M.M., Bronstein, A.M.: Shape recognition with spectral distances. PAMI 33(5), 1065–1071 (2010)CrossRefGoogle Scholar
  5. 5.
    Chatfield, K., Lempitsky, V.S., Vedaldi, A., Zisserman, A.: The devil is in the details: an evaluation of recent feature encoding methods. In: BMVC (2011)Google Scholar
  6. 6.
    Chen, X., Golovinskiy, A., Funkhouser, T.: A benchmark for 3D mesh segmentation. ACM TOG 28, 73 (2009)Google Scholar
  7. 7.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. PAMI 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  8. 8.
    Golovinskiy, A., Funkhouser, T.: Consistent segmentation of 3D models. Comput. Graph. 33(3), 262–269 (2009)CrossRefGoogle Scholar
  9. 9.
    Hu, R., Fan, L., Liu, L.: Co-segmentation of 3D shapes via subspace clustering. CGF 31, 1703–1713 (2012)Google Scholar
  10. 10.
    Huang, Q., Koltun, V., Guibas, L.: Joint shape segmentation with linear programming. ACM TOG 30, 125 (2011)Google Scholar
  11. 11.
    Huang, Q., Wang, F., Guibas, L.: Functional map networks for analyzing and exploring large shape collections. TOG 33(4), 36 (2014)Google Scholar
  12. 12.
    Jaakkola, T., Haussler, D., et al.: Exploiting generative models in discriminative classifiers. In: NIPS, pp. 487–493 (1999)Google Scholar
  13. 13.
    van Kaick, O., Tagliasacchi, A., Sidi, O., Zhang, H., Cohen-Or, D., Wolf, L., Hamarneh, G.: Prior knowledge for part correspondence. CGF 30, 553–562 (2011)Google Scholar
  14. 14.
    van Kaick, O., Xu, K., Zhang, H., Wang, Y., Sun, S., Shamir, A., Cohen-Or, D.: Co-hierarchical analysis of shape structures. TOG 32(4), 69 (2013)zbMATHGoogle Scholar
  15. 15.
    Kalogerakis, E., Hertzmann, A., Singh, K.: Learning 3D mesh segmentation and labeling. ACM TOG 29, 102 (2010)CrossRefGoogle Scholar
  16. 16.
    Kim, V.G., Li, W., Mitra, N.J., Chaudhuri, S., DiVerdi, S., Funkhouser, T.: Learning part-based templates from large collections of 3D shapes. TOG 32(4), 70 (2013)zbMATHGoogle Scholar
  17. 17.
    Liang, J., Lai, R., Wong, T.W., Zhao, H.: Geometric understanding of point clouds using Laplace-Beltrami operator. In: CVPR, pp. 214–221 (2012)Google Scholar
  18. 18.
    Liu, Y., Prabhakaran, B., Guo, X.: Point-based manifold harmonics. VCG 18(10), 1693–1703 (2012)Google Scholar
  19. 19.
    Marton, Z.C., Rusu, R.B., Beetz, M.: On fast surface reconstruction methods for large and noisy datasets. In: ICRA, Kobe, Japan, 12–17 May 2009Google Scholar
  20. 20.
    Meng, M., Xia, J., Luo, J., He, Y.: Unsupervised co-segmentation for 3D shapes using iterative multi-label optimization. CAD 45(2), 312–320 (2013)MathSciNetGoogle Scholar
  21. 21.
    Mitra, N.J., Wand, M., Zhang, H., Cohen-Or, D., Kim, V., Huang, Q.X.: Structure-aware shape processing. In: Eurographics Report. ACM (2014)Google Scholar
  22. 22.
    Ohbuchi, R., Osada, K., Furuya, T., Banno, T.: Salient local visual features for shape-based 3D model retrieval. In: SMA, pp. 93–102. IEEE (2008)Google Scholar
  23. 23.
    Ovsjanikov, M., Ben-Chen, M., Solomon, J., Butscher, A., Guibas, L.: Functional maps: a flexible representation of maps between shapes. TOG 31(4), 30 (2012)CrossRefGoogle Scholar
  24. 24.
    Ovsjanikov, M., Bronstein, A.M., Bronstein, M.M., Guibas, L.J.: Shape Google: a computer vision approach to isometry invariant shape retrieval. In: ICCV Workshops, pp. 320–327. IEEE (2009)Google Scholar
  25. 25.
    Ovsjanikov, M., Li, W., Guibas, L., Mitra, N.J.: Exploration of continuous variability in collections of 3D shapes. TOG 30(4), 33 (2011)CrossRefGoogle Scholar
  26. 26.
    Perronnin, F., Sánchez, J., Mensink, T.: Improving the fisher kernel for large-scale image classification. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 143–156. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  27. 27.
    Rusu, R.B., Blodow, N., Beetz, M.: Fast point feature histograms (FPFH) for 3D registration. In: ICRA, pp. 3212–3217. IEEE (2009)Google Scholar
  28. 28.
    Sánchez, J., Perronnin, F., Mensink, T., Verbeek, J.: Image classification with the fisher vector: theory and practice. IJCV 105(3), 222–245 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Schnitman, Y., Caspi, Y., Cohen-Or, D., Lischinski, D.: Inducing semantic segmentation from an example. In: Narayanan, P.J., Nayar, S.K., Shum, H.-Y. (eds.) ACCV 2006. LNCS, vol. 3852, pp. 373–384. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  30. 30.
    Schoeler, M., Papon, J., Wörgötter, F.: Constrained planar cuts-object partitioning for point clouds. In: CVPR, pp. 5207–5215 (2015)Google Scholar
  31. 31.
    Shapira, L., Shalom, S., Shamir, A., Cohen-Or, D., Zhang, H.: Contextual part analogies in 3D objects. IJCV 89(2), 309–326 (2010)CrossRefGoogle Scholar
  32. 32.
    Shapira, L., Shamir, A., Cohen-Or, D.: Consistent mesh partitioning and skeletonisation using the shape diameter function. VC 24(4), 249–259 (2008)Google Scholar
  33. 33.
    Shotton, J., Winn, J., Rother, C., Criminisi, A.: Textonboost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. IJCV 81(1), 2–23 (2009)CrossRefGoogle Scholar
  34. 34.
    Sidi, O., van Kaick, O., Kleiman, Y., Zhang, H., Cohen-Or, D.: Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. ACM TOG 30, 126 (2011)CrossRefGoogle Scholar
  35. 35.
    Toldo, R., Castellani, U., Fusiello, A.: Visual vocabulary signature for 3D object retrieval and partial matching. In: Proceedings of the 3DOR Conference, pp. 21–28 (2009)Google Scholar
  36. 36.
    Tombari, F., Salti, S., Di Stefano, L.: Unique signatures of histograms for local surface description. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 356–369. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  37. 37.
    Toshev, A., Shi, J., Daniilidis, K.: Image matching via saliency region correspondences. In: CVPR, pp. 1–8 (2007)Google Scholar
  38. 38.
    Van Kaick, O., Zhang, H., Hamarneh, G., Cohen-Or, D.: A survey on shape correspondence. CGF 30, 1681–1707 (2011)Google Scholar
  39. 39.
    Wainwright, M.J., Jaakkola, T.S., Willsky, A.S.: Map estimation via agreement on trees: message-passing and linear programming. IEEE Trans. Inf. Theor. 51(11), 3697–3717 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Wang, F., Huang, Q., Ovsjanikov, M., Guibas, L.J.: Unsupervised multi-class joint image segmentation. In: CVPR, pp. 3142–3149 (2014)Google Scholar
  41. 41.
    Wang, Y., Asafi, S., van Kaick, O., Zhang, H., Cohen-Or, D., Chen, B.: Active co-analysis of a set of shapes. TOG 31(6), 165 (2012)CrossRefGoogle Scholar
  42. 42.
    Xu, K., Li, H., Zhang, H., Cohen-Or, D., Xiong, Y., Cheng, Z.Q.: Style-content separation by anisotropic part scales. TOG 29(6), 184 (2010)CrossRefGoogle Scholar
  43. 43.
    Yedidia, J.S., Freeman, W.T., Weiss, Y.: Constructing free-energy approximations and generalized belief propagation algorithms. IEEE Trans. Inf. Theor. 51(7), 2282–2312 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Zhang, H., Sheffer, A., Cohen-Or, D., Zhou, Q., Van Kaick, O., Tagliasacchi, A.: Deformation-driven shape correspondence. CGF 27, 1431–1439 (2008)Google Scholar
  45. 45.
    Zheng, Y., Cohen-Or, D., Averkiou, M., Mitra, N.J.: Recurring part arrangements in shape collections. CGF 33, 115–124 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Nikita Araslanov
    • 1
    Email author
  • Seongyong Koo
    • 1
  • Juergen Gall
    • 1
  • Sven Behnke
    • 1
  1. 1.Computer Science InstituteUniversity of BonnBonnGermany

Personalised recommendations