Skip to main content
SpringerLink
Log in

3D model retrieval via single image based on feature mapping

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the development of manufacture, more and more 3D models are generated by users and many differnet factories. 3D model retrieval has been receiving more and more attention in computer vision and the field of data analysis. In this paper, we propose a novel 3D model retrieval algorithm by cross-modal feature mapping (CMFM), which utilize one single image as query information to address 3D model retrieval problem. Specifically, in this paper, we first proposed to leverage 2D image to handle 3d model retrieval problem, which is one new problem in this field. The proposed feature learning method can benefit: 1) avoiding the interference of query image recorded by different visual sensor; 2) handling cross-modal data retrieval by simple computer vision technologies, which can guarantee the performance of retrieval and also control that the retrieval time hold a low level; 3) the low complexity of this method can guarantee that this method can be applied in many fields. Finally, we validate the retrieval method on three popular datasets. Extensive comparison experiments show the superiority of the proposed mehtod. To the best of our knowledge, it is the first method to handle 3D model retreival based on one single 2D image.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ankerst M, Gabi Ller K, Kriegel HP, Seidl T (1999) 3d shape histograms for similarity search and classification in spatial databases. Lect Notes Comput Sci 1651:207–226

    Article  Google Scholar 

  2. Ansary TF, Daoudi M, Jean Philippe V (2007) A bayesian 3-d search engine using adaptive views clustering. IEEE Trans Multimed 9(1):78–88

    Article  Google Scholar 

  3. Arsigny V, Fillard P, Pennec X, Ayache N (2006) Geometric means in a novel vector space structure on symmetric positive-definite matrices. Siam J Matrix Anal Appl 29(1):328–347

    Article  MathSciNet  MATH  Google Scholar 

  4. Aubry M, Schlickewei U, Cremers D (2011) The wave kernel signature: A quantum mechanical approach to shape analysis. In: IEEE international conference on computer vision workshops. ICCV 2011 Workshops, Barcelona, pp 1626–1633

  5. Baumgart BG (2014) Geometric modeling for computer vision. Comp. Sci. 58 (1):85–135

    Google Scholar 

  6. Brennecke A, Isenberg T (2004) 3d shape matching using skeleton graphs. In: Simulation Und Visualisierung, pp 299–310

  7. Bustos B (2005) Feature-based similarity search in 3d object databases. Acm Comput Surv 37(4):345–387

    Article  Google Scholar 

  8. Chang S, Zhong Y, Quan Z, Hong Y, Zeng J, Du D (2016) A real-time object tracking and image stabilization system for photographing in vibration environment using opentld algorithm. In: 2016 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO), pp 141–145

  9. Chen DY, Tian XP, Yu TS, Ming O (2003) On visual similarity based 3d model retrieval. Comput Graph Forum 22(3):223–232

    Article  Google Scholar 

  10. Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27

    Article  MATH  Google Scholar 

  11. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 886–893

  12. Daras P, Axenopoulos A (2010) A 3d shape retrieval framework supporting multimodal queries. Int J Comput Vis 89(2):229–247

    Article  Google Scholar 

  13. Davis JV, Kulis B, Jain P, Sra S, Dhillon IS (2007) Information-theoretic metric learn. In: Machine Learning Proceedings of the Twenty-Fourth International Conference, pp 209–216

  14. Duchenne O, Bach F, In SK, Ponce J (2011) A tensor-based algorithm for high-order graph matching. IEEE Trans Pattern Anal Mach Intell 33(12):2383–95

    Article  Google Scholar 

  15. Fang Y, Xie J, Dai G, Wang M, Zhu F, Xu T, Wong E (2015) 3d deep shape descriptor. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2319–2328

  16. Funkhouser T, Min P, Kazhdan M, Chen J, Halderman A, Dobkin D, Jacobs D (2003) A search engine for 3d models. Acm Trans Graph 22(1):83–105

    Article  Google Scholar 

  17. Gao Y, Wang M, Tao D, Ji R, Dai Q (2012) 3-d object retrieval and recognition with hypergraph analysis. IEEE Trans Image Process Publ IEEE Signal Process Soc 21(9):4290–303

    Article  MathSciNet  MATH  Google Scholar 

  18. Gao Y, Dai Q, Wang M, Zhang N (2011) 3d model retrieval using weighted bipartite graph matching. Signal Process Image Commun 26(1):39–47

    Article  Google Scholar 

  19. Gao Y, Dai Q, Zhang NY (2010) 3d model comparison using spatial structure circular descriptor. Pattern Recogn 43(3):1142–1151

    Article  MATH  Google Scholar 

  20. Gao Y, Tang J, Hong R, Yan S (2012) Camera constraint-free view-based 3-d object retrieval. IEEE Trans Image Process Publ IEEE Signal Process Soc 21(4):2269–2281

    Article  MathSciNet  MATH  Google Scholar 

  21. Gao Z, Zhang H, Xu GP, Xue YB (2015) Multi-perspective and multi-modality joint representation and recognition model for 3d action recognition. Neurocomputing 151:554–564

    Article  Google Scholar 

  22. Gao Z, Nie W, Liu A, Zhang H (2016) Evaluation of local spatial–temporal features for cross-view action recognition. Neurocomputing 173:110–117

    Article  Google Scholar 

  23. Gao Z, Zhang H, Xu GP, Xue YB, Hauptmann AG (2015) Multi-view discriminative and structured dictionary learning with group sparsity for human action recognition. Signal Process 112:83–97

    Article  Google Scholar 

  24. Gao Z, Zhang L-f, Chen M-y, Hauptmann A, Zhang H, Cai A-N (2014) Enhanced and hierarchical structure algorithm for data imbalance problem in semantic extraction under massive video dataset. Multimed Tools Appl 68(3):641–657

    Article  Google Scholar 

  25. Goldberger J, Roweis ST, Hinton GE, Salakhutdinov R (2004) Neighbourhood components analysis. Adv Neural Inf Process Syst 83(6):513–520

    Google Scholar 

  26. Guétat G, Maitre M, Joly L, Lai SL, Lee T, Shinagawa Y (2006) Automatic 3-d grayscale volume matching and shape analysis. IEEE Trans Inf Technol Biomed Publ IEEE Eng Med Biol Soc 10(2):362–76

    Article  Google Scholar 

  27. Hsieh CT, Han CC, Shih JL, Lee CH (2015) 3d model retrieval using multiple features and manifold ranking. In: International conference on ubi-media computing

  28. Ip CY, Lapadat D, Sieger L, Regli WC (2002) Using shape distributions to compare solid models. In: ACM symposium on solid modeling and applications, pp 273–280

  29. Kim WY, Kim YS (2000) A region-based shape descriptor using zernike moments. Signal Process Image Commun 16(1-2):95–102

    Article  Google Scholar 

  30. Leibe B, Schiele B (2003) Analyzing appearance and contour based methods for object categorization. In: 2003. Proceedings. 2003 IEEE computer society conference on computer vision and pattern recognition, pp 409–415

  31. Mohamed W, Ben Hamza A (2012) Reeb graph path dissimilarity for 3d object matching and retrieval. Vis Comput 28(3):305–318

    Article  Google Scholar 

  32. Nie WZ, An AL, Gao Z, Su YT (2015) Clique-graph matching by preserving global and local structure, pp 4503–4510

  33. Nie W, Cao Q, Liu A, Su Y (2017) Convolutional deep learning for 3D object retrieval. Multimedia Syst 23(3):325–332

  34. Nie W, Li X, Liu A, Su Y (2017) 3D object retrieval based on Spatial+LDA model. Multimed Tools Appl 76(3):4091–410

  35. Novatnack J, Nishino K (2007) Scale-dependent 3d geometric features, pp 1–8

  36. Ohbuchi R, Osada K, Furuya T, Banno T (2008) Salient local visual features for shape-based 3d model retrieval. In: IEEE international conference on shape modeling and applications, pp 93–102

  37. Ohbuchi R, Furuya T (2009) Scale-weighted dense bag of visual features for 3d model retrieval from a partial view 3d model. In: IEEE International Conference on Computer Vision Workshops, pp 63–70

  38. Osada R, Funkhouser T, Chazelle B, Dobkin D (2001) 3d models with shape distributions. In: SMI 2001 International Conference on Shape Modeling and Applications, pp 0154–0154

  39. Osada R, Funkhouser T, Chazelle B, Dobkin D (2002) Shape distributions. Acm Trans Graph 21(4):807–832

    Article  MathSciNet  MATH  Google Scholar 

  40. Papoiu AD, Emerson NM, Patel TS, Kraft RA, Valdes-Rodriguez R, Nattkemper LA, Coghill RC, Yosipovitch G (2014) Voxel-based morphometry and arterial spin labeling fmri reveal neuropathic and neuroplastic features of brain processing of itch in end-stage renal disease. J Neurophysiol 112(7):1729–38

    Article  Google Scholar 

  41. Paquet E, Rioux M, Murching A, Naveen T, Tabatabai A (2000) Description of shape information for 2-d and 3-d objects. Signal Process Image Commun 16(s 1–2):103–122

    Article  Google Scholar 

  42. Persoon E, Fu KS (1986) Shape discrimination using fourier descriptors. IEEE Trans Syst Cybern 7(3):170–179

    Article  MathSciNet  Google Scholar 

  43. Polewski P, Yao W, Heurich M, Krzystek P, Stilla U (2015) Active learning approach to detecting standing dead trees from ALS point clouds combined with aerial infrared imagery. CVPRWorkshop, pp 10–18

  44. Regli WC, Cicirello VA (2000) Managing digital libraries for computer-aided design. Comput-Aided Des 32(2):119–132

    Article  Google Scholar 

  45. Sharma A, Jacobs DW (2011) Bypassing synthesis: Pls for face recognition with pose, low-resolution and sketch. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 593–600

  46. Shih JL, Lee CH, Wang JT (2007) A new 3d model retrieval approach based on the elevation descriptor. Pattern Recogn 40(1):283–295

    Article  MATH  Google Scholar 

  47. Shinagawa Y, Kunii TL (1991) Constructing a reeb graph automatically from cross sections. IEEE Comput Graph Appl 11(6):44–51

    Article  Google Scholar 

  48. Sundar H, Silver D, Gagvani N, Dickinson S (2003) Skeleton based shape matching and retrieval. In: Shape modeling international, p 130

  49. Tangelder JWH, Veltkamp RC (2003) Polyhedral model retrieval using weighted point sets. In: Shape modeling international, pp 209–229

  50. Vavilov D, Dovzhenko D, Anisimov A (2010) Perspectives of stereo 3d tv applications development. In: Software engineering conference, pp 175–178

  51. Vinyals O, Toshev A, Bengio S, Erhan D (2015) Show and tell: A neural image caption generator. In: Computer vision and pattern recognition, pp 3156–3164

  52. Wang Y, Liu Z, Pang F, Li H (2015) Boosting 3d model retrieval with class vocabularies and distance vector revision, pp 1–5

  53. Yeh JS, Chen DY, Chen BY, Ouhyoung M (2005) A web-based three-dimensional protein retrieval system by matching visual similarity. Bioinformatics 21(13):3056–3057

    Article  Google Scholar 

  54. Zhao S, Yao H, Zhang Y, Wang Y, Liu S (2015) View-based 3d object retrieval via multi-modal graph learning. Signal Process 112(C):110–118

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic Information Engineering, Tianjin University, Tianjin, 300072, China

    Anan Liu, Nannan Liu, Weizhi Nie & Yuting Su

Authors
  1. Anan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nannan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weizhi Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuting Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weizhi Nie.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, A., Liu, N., Nie, W. et al. 3D model retrieval via single image based on feature mapping. Multimed Tools Appl 77, 22051–22069 (2018). https://doi.org/10.1007/s11042-017-5271-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5271-z

Keywords

Search

Navigation