Skip to main content
SpringerLink
Log in

Deep Learning for 3D Data Processing

  • Chapter
Deep Learning in Object Detection and Recognition

  • 3230 Accesses

Abstract

Extracting local features from raw 3D data is a nontrivial and challenging task that requires carefully designed 3D shape descriptors. In conventional methods, these descriptors are handcrafted and require intensive human intervention and prior knowledge. To tackle this issue, we propose a novel deep learning model, namely, Circle Convolutional Restricted Boltzmann Machine (CCRBM), for unsupervised 3D local feature learning. CCRBM is specially designed for 3D shapes which effectively resolves the obstacles in the hierarchical learning process that existing deep learning models cannot resolve, such as irregular topology of vertices, orientation ambiguity on the 3D surface, and rigid or slightly nonrigid transformation invariance. Specially, by introducing the novel circle convolution, CCRBM holds a novel ring-like multilayer structure to learn 3D local features in a manner of structure preservation. Circle convolution convolves across 3D local regions with a novel circular sector convolution window by rotating itself along a xed circle direction. In the process of circle convolution, extra points are sampled on each 3D local region and projected onto the tangent plane of the center of the region. By this way, the projection distances in each sector window are employed to constitute the raw 3D feature called projection distance distribution (PDD). In addition, to eliminate the ambiguity of the initial location of a sector window, Fourier Transform Modulus (FTM) is used to transform the PDD into Fourier domain which is then conveyed to CCRBM. Experiments using the learned local features are conducted on three aspects: global shape retrieval, partial shape retrieval, and shape correspondence. The experimental results show that the learned local features outperform other state-of-the-art 3D shape descriptors.

This work was supported by the Natural Science Foundation of China under Grant 61672430, 61573284 and 61522207, and NWPU Basic Research Fund under Grant 3102016JKBJJGZ08.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Y. Gao, M. Wang, D. Tao, R. Ji, and Q. Dai, 3D object retrieval and recognition with hypergraph analysis, IEEE Transactions on Image Processing, vol. 21, no. 9, pp. 4290–4303, 2012.

    Article  MathSciNet  Google Scholar 

  2. O. van Kaick, H. Zhang, G. Hamarneh, and D. Cohen-Or, A survey on shape correspondence, Computer Graphics Forum, vol. 30, no. 6, pp. 1681–1707, Sep 2011.

    Article  Google Scholar 

  3. X. Qian and C. Ye, Ncc-ransac: A fast plane extraction method for 3D range data segmentation, IEEE Transactions on Cybernetics, vol. 44, pp. 2771–2783, 2014.

    Article  Google Scholar 

  4. K.-C. Chan, C.-K. Koh, and C. S. G. Lee, A 3-d-point-cloud system for human-pose estimation, IEEE Transactions on Cybernetics, vol. 44, pp. 1486–1497, 2014.

    Google Scholar 

  5. O. S. Gedik and A. A. Alatan, 3D rigid body tracking using vision and depth sensors, IEEE Transaction on Cybernetics, vol. 43, no. 5, pp. 1395–1405, 2013.

    Article  Google Scholar 

  6. M. Liang, H. Min, R. Luo, and J. Zhu, Simultaneous recognition and modeling for learning 3D object models from everyday scenes, IEEE Transactions on Cybernetics, vol. PP, pp. 1–12, 2014.

    Google Scholar 

  7. F.-F. Li and P. Perona, A bayesian hierarchical model for learning natural scene categories, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2005, pp. 524–531.

    Google Scholar 

  8. A. M. Bronstein, M. M. Bronstein, L. J. Guibas, and M. Ovsjanikov, Shape google: geometric words and expressions for invariant shape retrieval, ACM Transactions on Graphics, vol. 30, no. 1, pp. 1–20, 2011.

    Article  Google Scholar 

  9. L. Shapira, A. Shamir, and D. Cohen-Or, Consistent mesh partitioning and skeletonisation using the shape diameter function, The Visual Computer, vol. 24, no. 4, pp. 249–259, 2008.

    Article  Google Scholar 

  10. A. Johnson and M. Hebert, Using spin images for efficient object recognition in cluttered 3D scenes, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 5, pp. 433–449, 1999.

    Article  Google Scholar 

  11. T. Darom and Y. Keller, Scale-invariant features for 3D mesh models. IEEE Transactions on Image Processing, vol. 21, no. 5, pp. 2758–2769, 2012.

    Article  MathSciNet  Google Scholar 

  12. Y. Bengio, A. Courville, and P. Vincent, Representation learning: A review and new perspectives, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1798–1828, 2013.

    Article  Google Scholar 

  13. H. Yan, J. Lu, and X. Zhou, Prototype-based discriminative feature learning for kinship verification, IEEE Transactions on Cybernetics, vol. PP, pp. 1–13, 2014.

    Google Scholar 

  14. X. Lu, Y. Yuan, and P. Yan, Alternatively constrained dictionary learning for image superresolution, IEEE Transactions on Cybernetics, vol. 44, pp. 366–377, 2014.

    Article  Google Scholar 

  15. H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng, Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations, in The Annual International Conference on Machine Learning, 2009, pp. 609–616.

    Google Scholar 

  16. A. Krizhevsky, I. Sutskever, and G. E. Hinton, Imagenet classification with deep convolutional neural networks, in Advances in Neural Information Processing Systems, 2012, vol. 25, pp. 1097–1105.

    Google Scholar 

  17. A. Graves, A. rahman Mohamed, and G. E. Hinton, Speech recognition with deep recurrent neural networks, in IEEE International Conference on Acoustics, Speech, and Signal Processing, 2013, pp. 6645–6649.

    Google Scholar 

  18. S. Bu, Z. Liu, J. Han, J. Wu, and R. Ji, Learning high-level feature by deep belief networks for 3D model retrieval and recognition, IEEE Transactions on Multimedia, vol. 16, no. 8, pp. 2154–2167, 2014.

    Article  Google Scholar 

  19. Z. Liu, S. Chen, S. Bu, and K. Li, High-level semantic feature for 3D shape based on deep belief networks, Proceeding of IEEE International Conference on Multimedia and Expo, 2014, pp. 1–6.

    Google Scholar 

  20. B. Leng, X. Zhang, M. Yao, and Z. Xiong, A 3D model recognition mechanism based on deep boltzmann machines, Neurocomputing, vol. 151, pp. 593–602, 2014.

    Article  Google Scholar 

  21. P. Heider, A. Pierre-Pierre, R. Li, and C. Grimm, Local shape descriptors, a survey and evaluation, in Eurographics Workshop on 3D Object Retrieval, 2011, pp. 49–57.

    Google Scholar 

  22. J. Lodder, Curvature in the calculus curriculum. The American Mathematical Monthly, vol. 110, no. 7, pp. 593–605, 2003.

    Article  MathSciNet  Google Scholar 

  23. J. Sun, M. Ovsjanikov, and L. J. Guibas, A concise and provably informative multi-scale signature based on heat diffusion, Computer Graphics Forum, vol. 28, no. 5, pp. 1383–1392, 2009.

    Article  Google Scholar 

  24. M. M. Bronstein and I. Kokkinos, Scale-invariant heat kernel signatures for non-rigid shape recognition, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2010, pp. 1704–1711.

    Google Scholar 

  25. M. Aubry, U. Schlickewei, and D. Cremers, The wave kernel signature: A quantum mechanical approach to shape analysis, in International Conference on Computer Vision Workshops, 2011, pp. 1626–1633.

    Google Scholar 

  26. R. Socher, B. Huval, B. Bhat, C. D. Manning, and A. Y. Ng, Convolutional-recursive deep learning for 3D object classification, in Advances in Neural Information Processing Systems, 2012, vol. 25, pp. 665–673.

    Google Scholar 

  27. Z. Wu, S. Song, A. Khosla, X. Tang, and J. Xiao, 3D ShapeNets for 2.5D object recognition and Next-Best-View prediction, arXiv: 1406.5670, 2014.

    Google Scholar 

  28. G. E. Hinton, S. Osindero, and Y.-W. Teh, A fast learning algorithm for deep belief nets, Neural Computation, vol. 18, no. 7, pp. 1527–1554, 2006.

    Article  MathSciNet  Google Scholar 

  29. G. E. Hinton and R. R. Salakhutdinov, Reducing the dimensionality of data with neural networks, Science, vol. 313, pp. 504–507, 2006.

    Article  MathSciNet  Google Scholar 

  30. H. Ackley, E. Hinton, and J. Sejnowski, A learning algorithm for boltzmann machines, Cognitive Science, pp. 147–169, 1985.

    Google Scholar 

  31. G. E. Hinton, Training products of experts by minimizing contrastive divergence, Neural Computation, vol. 14, no. 8, pp. 1771–1800, 2002.

    Article  Google Scholar 

  32. G. Casella and E. I. George, Explaining the gibbs sampler, The American Statistician, vol. 46, no. 3, pp. 167–174, 1992.

    MathSciNet  Google Scholar 

  33. R. Osada, T. Funkhouser, B. Chazelle, and D. Dobkin, Shape distributions, ACM Transactions on Graphics, vol. 21, no. 4, pp. 807–832, 2002.

    Article  MathSciNet  Google Scholar 

  34. G. Peyre and L. D. Cohen, Geodesic remeshing using front propagation, International Journal of Computer Vision, vol. 69, no. 1, pp. 145–156, 2006.

    Article  Google Scholar 

  35. S. Zokai and G. Wolberg, Image registration using log-polar mappings for recovery of large-scale similarity and projective transformations, vol. 14, pp. 1422–1434, 2005.

    Google Scholar 

  36. I. Kokkinos and A. L. Yuille, Scale invariance without scale selection, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8.

    Google Scholar 

  37. I. Kokkinos, M. M. Bronstein, R. Litman, and A. M. Bronstein, Intrinsic shape context descriptors for deformable shapes, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 159–166.

    Google Scholar 

  38. K. Siddiqi, J. Zhang, D. Macrini, A. Shokoufandeh, S. Bouix, and S. Dickinson, Retrieving articulated 3D models using medial surfaces, Machine Vision and Applications, vol. 19, no. 4, pp. 261–275, 2008.

    Article  Google Scholar 

  39. D. Giorgi, S. Biasotti, and L. Paraboschi, Shape retrieval contest 2007: Watertight models track, 2007.

    Google Scholar 

  40. S. Marini, L. Paraboschi, and S. Biasotti, Shape retrieval contest 2007: Partial matching track, in SHREC in conjunction with IEEE Shape Modelling International, 2007, pp. 13–16.

    Google Scholar 

  41. D. Anguelov, P. Srinivasan, H.-C. Pang, D. Koller, S. Thrun, and J. Davis, The correlated correspondence algorithm for unsupervised registration of nonrigid surfaces, Proceeding of the Neural Information Processing Systems, 2004, pp. 33–40.

    Google Scholar 

  42. V. G. Kim, Y. Lipman, and T. Funkhouser, Blended intrinsic maps, ACM Transaction on Graphics, vol. 30, no. 4, pp. 79:1–79:12, 2011.

    Google Scholar 

  43. A. Elad and R. Kimmel, On bending invariant signatures for surfaces, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 10, pp. 1285–1295, 2003.

    Article  Google Scholar 

  44. R. Martin, W. F. Erich, and P. Niklas, Laplace-spectra as fingerprints for shape matching, in Symposium on Solid and Physical Modeling, 2005, pp. 101–106.

    Google Scholar 

  45. X. Wang, Y. Liu, and H. Zha, Intrinsic spin images: A subspace decomposition approach to understanding 3D deformable shapes, in 5th International Symposium 3D Data Processing, Visualization and Transmission, 2010, pp. 17–20.

    Google Scholar 

  46. D. Chen, X. Tian, Y. Shen, and M. Ouhyoung, On visual similarity based 3D model retrieval, Computer Graphics Forum, vol. 22, no. 3, pp. 223–232, 2003.

    Article  Google Scholar 

  47. M. Kazhdan, T. Funkhouser, and S. Rusinkiewicz, Rotation invariant spherical harmonic representation of 3D shape descriptors, Proceedings of Eurographics Symposium on Geometry Processing, 2003, pp. 156–165.

    Google Scholar 

  48. G. Lavou, Combination of Bag-of-Words Descriptors for Robust Partial Shape Retrieval, The Visual Computer, vol. 28, no. 9, pp. 931–942, 2012.

    Article  Google Scholar 

  49. H. Tabia, M. Daoudi, J.-P. Vandeborre, and O. Colot, A new 3D-matching method of non-rigid and partially similar models using curve analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 4, pp. 852–858, 2011.

    Article  Google Scholar 

  50. R. Toldo, U. Castellani, and A. Fusiello, Visual vocabulary signature for 3D object retrieval and partial matching. in Eurographics Workshop on 3D Object Retrieval, 2009, pp. 21–28.

    Google Scholar 

  51. H. Tabia, H. Laga, D. Picard, and P.-H. Gosselin, Covariance descriptors for 3D shape matching and retrieval, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 4815–4192, 2014.

    Google Scholar 

  52. S. Biasotti, S. Marini, M. Spagnuolo, and B. Falcidieno, Sub-part correspondence by structural descriptors of 3D shapes. Computer-Aided Design, vol. 38, pp. 1002–1019, 2006.

    Article  Google Scholar 

  53. N. D. Cornea, M. F. Demirci, D. Silver, A. Shokoufandeh, S. J. Dickinson, and P. B. Kantor, 3D object retrieval using many-to-many matching of curve skeletons, Proceedings of the International Conference on Shape Modeling and Applications, 2005, pp. 368–373.

    Google Scholar 

  54. J. Tierny, J.-P. Vandeborre, and M. Daoudi, Partial 3D shape retrieval by reeb pattern unfolding. Computer Graphics Forum, vol. 28, pp. 41–55, 2009.

    Article  Google Scholar 

  55. H. W. Kuhn, The hungarian method for the assignment problem, Naval Research Logistics Quarterly, vol. 2, pp. 83–97, 1955.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Northwestern Polytechnical University, Xi’an, China

    Zhenbao Liu, Zhizhong Han & Shuhui Bu

Authors
  1. Zhenbao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhizhong Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuhui Bu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuhui Bu .

Editor information

Editors and Affiliations

  1. School of Electronics and Information, Northwestern Polytechnical University, Xi’an, Shaanxi, China

    Xiaoyue Jiang  & Xiaoyi Feng  & 

  2. Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, Oulu, Finland

    Abdenour Hadid

  3. School of Electrical and Information Engineering, Tianjin University, Tianjin, Tianjin, China

    Yanwei Pang

  4. École de technologie supérieure, University of Québec, Montréal, QC, Canada

    Eric Granger

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this chapter

Cite this chapter

Liu, Z., Han, Z., Bu, S. (2019). Deep Learning for 3D Data Processing. In: Jiang, X., Hadid, A., Pang, Y., Granger, E., Feng, X. (eds) Deep Learning in Object Detection and Recognition. Springer, Singapore. https://doi.org/10.1007/978-981-10-5152-4_7

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-5152-4_7

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-5151-7

  • Online ISBN: 978-981-10-5152-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation