Learning Spectral Transform Network on 3D Surface for Non-rigid Shape Analysis

  • Ruixuan Yu
  • Jian SunEmail author
  • Huibin Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11131)


Designing a network on 3D surface for non-rigid shape analysis is a challenging task. In this work, we propose a novel spectral transform network on 3D surface to learn shape descriptors. The proposed network architecture consists of four stages: raw descriptor extraction, surface second-order pooling, mixture of power function-based spectral transform, and metric learning. The proposed network is simple and shallow. Quantitative experiments on challenging benchmarks show its effectiveness for non-rigid shape retrieval and classification, e.g., it achieved the highest accuracies on SHREC’14, 15 datasets as well as the “range” subset of SHREC’17 dataset.


Non-rigid shape analysis Spectral transform Shape representation 



This work is supported by National Natural Science Foundation of China under Grants 11622106, 61711530242, 61472313, 11690011, 61721002.

Supplementary material

478822_1_En_28_MOESM1_ESM.pdf (39 kb)
Supplementary material 1 (pdf 39 KB)


  1. 1.
    Arsigny, V., Fillard, P., Pennec, X., Ayache, N.: Geometric means in a novel vector space structure on symmetric positive-definite matrices. SIAM J. Matrix Anal. Appl. 29(1), 328–347 (2007)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Aubry, M., Schlickewei, U., Cremers, D.: The wave kernel signature: a quantum mechanical approach to shape analysis. In: ICCV, pp. 1626–1633 (2011)Google Scholar
  3. 3.
    Bai, S., Bai, X., Zhou, Z., Zhang, Z., Tian, Q., Latecki, L.J.: Gift: towards scalable 3D shape retrieval. IEEE Trans. Multimed. 19(6), 1257–1271 (2017)CrossRefGoogle Scholar
  4. 4.
    Belongie, S., Malik, J., Puzicha, J.: Shape context: a new descriptor for shape matching and object recognition. In: NIPS, pp. 831–837 (2001)Google Scholar
  5. 5.
    Boscaini, D., Masci, J., Rodolà, E., Bronstein, M.: Learning shape correspondence with anisotropic convolutional neural networks. In: NIPS, pp. 3189–3197 (2016)Google Scholar
  6. 6.
    Bronstein, M.M., Kokkinos, I.: Scale-invariant heat kernel signatures for non-rigid shape recognition. In: CVPR, pp. 1704–1711 (2010)Google Scholar
  7. 7.
    Carreira, J., Caseiro, R., Batista, J., Sminchisescu, C.: Semantic segmentation with second-order pooling. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7578, pp. 430–443. Springer, Heidelberg (2012). Scholar
  8. 8.
    Chiotellis, I., Triebel, R., Windheuser, T., Cremers, D.: Non-rigid 3D shape retrieval via large margin nearest neighbor embedding. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 327–342. Springer, Cham (2016). Scholar
  9. 9.
    Cirujeda, P., Mateo, X., Dicente, Y., Binefa, X.: MCOV: a covariance descriptor for fusion of texture and shape features in 3D point clouds. In: 3DV, pp. 551–558 (2015)Google Scholar
  10. 10.
    Dryden, I.L., Koloydenko, A., Zhou, D.: Non-euclidean statistics for covariance matrices, with applications to diffusion tensor imaging. Ann. Appl. Stat. 3(3), 1102–1123 (2009)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Elad, A., Kimmel, R.: On bending invariant signatures for surfaces. IEEE TPAMI 25(10), 1285–1295 (2003)CrossRefGoogle Scholar
  12. 12.
    Fang, Y., et al.: 3D deep shape descriptor. In: CVPR, pp. 2319–2328 (2015)Google Scholar
  13. 13.
    Furuya, T., Ohbuchi, R.: Deep aggregation of local 3D geometric features for 3D model retrieval. In: BMVC, pp. 121.1–121.12 (2016)Google Scholar
  14. 14.
    Gasparetto, A., Torsello, A.: A statistical model of Riemannian metric variation for deformable shape analysis. In: CVPR, pp. 1219–1228 (2015)Google Scholar
  15. 15.
    Huang, Z., Van Gool, L.: A Riemannian network for SPD matrix learning. In: AAAI (2017)Google Scholar
  16. 16.
    Ionescu, C., Carreira, J., Sminchisescu, C.: Iterated second-order label sensitive pooling for 3D human pose estimation. In: CVPR, pp. 1661–1668 (2014)Google Scholar
  17. 17.
    Ionescu, C., Vantzos, O., Sminchisescu, C.: Matrix backpropagation for deep networks with structured layers. In: ICCV, pp. 2965–2973 (2015)Google Scholar
  18. 18.
    Jegou, H., Douze, M., Schmid, C., Perez, P.: Aggregating local descriptors into a compact image representation. In: CVPR, pp. 3304–3311 (2010)Google Scholar
  19. 19.
    Johnson, A.E., Hebert, M.: Using spin images for efficient object recognition in cluttered 3D scenes. IEEE TPAMI 21(5), 433–449 (1999)CrossRefGoogle Scholar
  20. 20.
    Lafon, S., Keller, Y., Coifman, R.R.: Data fusion and multicue data matching by diffusion maps. IEEE TPAMI 28(11), 1784–1797 (2006)CrossRefGoogle Scholar
  21. 21.
    Li, B., Lu, Y., Li, C., et al.: A comparison of 3D shape retrieval methods based on a large-scale benchmark supporting multimodal queries. CVIU 131(c), 1–27 (2015)Google Scholar
  22. 22.
    Li, H., Huang, D., Morvan, J., Wang, Y., Chen, L.: Towards 3D face recognition in the real: a registration-free approach using fine-grained matching of 3D keypoint descriptors. IJCV 113(2), 128–142 (2015)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Lian, Z., Zhang, J., et al.: Shrec’15 track: non-rigid 3D shape retrieval. In: Eurographics 3DOR Workshop (2015)Google Scholar
  24. 24.
    Lin, T.Y., RoyChowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition. In: ICCV, pp. 1449–1457 (2015)Google Scholar
  25. 25.
    Litman, R., Bronstein, A., Bronstein, M., Castellani, U.: Supervised learning of bag-of-features shape descriptors using sparse coding. CGF 33(5), 127–136 (2014)Google Scholar
  26. 26.
    Luciano, L., Hamza, A.B.: Deep learning with geodesic moments for 3D shape classification. Pattern Recogn. Lett. 83, 339–348 (2017)Google Scholar
  27. 27.
    Masci, J., Boscaini, D., Bronstein, M., Vandergheynst, P.: Geodesic convolutional neural networks on Riemannian manifolds. In: ICCV, pp. 832–840 (2015)Google Scholar
  28. 28.
    Masoumi, M., Rodola, E., Cosmo, L.: Shrec’17 track: Deformable shape retrieval with missing parts. In: Eurographics 3DOR Workshop (2017)Google Scholar
  29. 29.
    Ohkita, Y., Ohishi, Y., Furuya, T., Ohbuchi, R.: Non-rigid 3D model retrieval using set of local statistical features. In: IEEE International Conference on Multimedia and Expo Workshops, pp. 593–598 (2012)Google Scholar
  30. 30.
    Osada, R., Funkhouser, T., Chazelle, B., Dobkin, D.: Shape distributions. ACM TOG 21(4), 807–832 (2002)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Pennec, X., Fillard, P., Ayache, N.: A Riemannian framework for tensor computing. IJCV 66(1), 41–66 (2006)CrossRefGoogle Scholar
  32. 32.
    Pickup, D., Sun, X., Rosin, P.L., et al.: Shrec’14 track: shape retrieval of non-rigid 3D human models. In: Eurographics 3DOR Workshop (2014)Google Scholar
  33. 33.
    Pinkall, U., Polthier, K.: Computing discrete minimal surfaces and their conjugates. Exp. Math. 2(1), 15–36 (1993)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: deep learning on point sets for 3D classification and segmentation. In: CVPR, pp. 77–85 (2017)Google Scholar
  35. 35.
    Qi, C.R., Su, H., Nießner, M., Dai, A., Yan, M., Guibas, L.J.: Volumetric and multi-view CNNs for object classification on 3D data. In: CVPR, pp. 5648–5656 (2016)Google Scholar
  36. 36.
    Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: NIPS, pp. 5099–5108 (2017)Google Scholar
  37. 37.
    Reuter, M., Wolter, F., Peinecke, N.: Laplace-Beltrami spectra as ‘Shape-DNA’ of surfaces and solids. Comput. Aided Des. 38(4), 342–366 (2006)CrossRefGoogle Scholar
  38. 38.
    Shilane, P., Min, P., Kazhdan, M., Funkhouser, T.: The Princeton shape benchmark. In: IEEE International Conference on Shape Modeling and Applications, pp. 167–178 (2004)Google Scholar
  39. 39.
    Smeets, D., Keustermans, J., Vandermeulen, D., Suetens, P.: meshSIFT: local surface features for 3D face recognition under expression variations and partial data. CVIU 117(2), 158–169 (2013)Google Scholar
  40. 40.
    Su, H., Maji, S., Kalogerakis, E., Learnedmiller, E.G.: Multi-view convolutional neural networks for 3D shape recognition. In: ICCV, pp. 945–953 (2015)Google Scholar
  41. 41.
    Sun, J., Ovsjanikov, M., Guibas, L.: A concise and provably informative multi-scale signature based on heat diffusion. In: CGF, pp. 1383–1392 (2009)Google Scholar
  42. 42.
    Szegedy, C., et al.: Going deeper with convolutions. In: CVPR, pp. 1–9 (2015)Google Scholar
  43. 43.
    Tabia, H., Laga, H., Picard, D., Gosselin, P.H.: Covariance descriptors for 3D shape matching and retrieval. In: CVPR, pp. 4185–4192 (2014)Google Scholar
  44. 44.
    Wu, Z., et al.: 3D shapenets: a deep representation for volumetric shapes. In: CVPR, pp. 1912–1920 (2015)Google Scholar
  45. 45.
    Zhu, F., Xie, J., Fang, Y.: Heat diffusion long-short term memory learning for 3D shape analysis. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 305–321. Springer, Cham (2016). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Xi’an Jiaotong UniversityXi’anChina

Personalised recommendations