Multimedia Tools and Applications

, Volume 77, Issue 17, pp 22051–22069 | Cite as

3D model retrieval via single image based on feature mapping

  • Anan Liu
  • Nannan Liu
  • Weizhi NieEmail author
  • Yuting Su


With the development of manufacture, more and more 3D models are generated by users and many differnet factories. 3D model retrieval has been receiving more and more attention in computer vision and the field of data analysis. In this paper, we propose a novel 3D model retrieval algorithm by cross-modal feature mapping (CMFM), which utilize one single image as query information to address 3D model retrieval problem. Specifically, in this paper, we first proposed to leverage 2D image to handle 3d model retrieval problem, which is one new problem in this field. The proposed feature learning method can benefit: 1) avoiding the interference of query image recorded by different visual sensor; 2) handling cross-modal data retrieval by simple computer vision technologies, which can guarantee the performance of retrieval and also control that the retrieval time hold a low level; 3) the low complexity of this method can guarantee that this method can be applied in many fields. Finally, we validate the retrieval method on three popular datasets. Extensive comparison experiments show the superiority of the proposed mehtod. To the best of our knowledge, it is the first method to handle 3D model retreival based on one single 2D image.


3D model retrieval View-based Feature mapping/learning Iteration optimization CNN 


  1. 1.
    Ankerst M, Gabi Ller K, Kriegel HP, Seidl T (1999) 3d shape histograms for similarity search and classification in spatial databases. Lect Notes Comput Sci 1651:207–226CrossRefGoogle Scholar
  2. 2.
    Ansary TF, Daoudi M, Jean Philippe V (2007) A bayesian 3-d search engine using adaptive views clustering. IEEE Trans Multimed 9(1):78–88CrossRefGoogle Scholar
  3. 3.
    Arsigny V, Fillard P, Pennec X, Ayache N (2006) Geometric means in a novel vector space structure on symmetric positive-definite matrices. Siam J Matrix Anal Appl 29(1):328–347MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Aubry M, Schlickewei U, Cremers D (2011) The wave kernel signature: A quantum mechanical approach to shape analysis. In: IEEE international conference on computer vision workshops. ICCV 2011 Workshops, Barcelona, pp 1626–1633Google Scholar
  5. 5.
    Baumgart BG (2014) Geometric modeling for computer vision. Comp. Sci. 58 (1):85–135Google Scholar
  6. 6.
    Brennecke A, Isenberg T (2004) 3d shape matching using skeleton graphs. In: Simulation Und Visualisierung, pp 299–310Google Scholar
  7. 7.
    Bustos B (2005) Feature-based similarity search in 3d object databases. Acm Comput Surv 37(4):345–387CrossRefGoogle Scholar
  8. 8.
    Chang S, Zhong Y, Quan Z, Hong Y, Zeng J, Du D (2016) A real-time object tracking and image stabilization system for photographing in vibration environment using opentld algorithm. In: 2016 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO), pp 141–145Google Scholar
  9. 9.
    Chen DY, Tian XP, Yu TS, Ming O (2003) On visual similarity based 3d model retrieval. Comput Graph Forum 22(3):223–232CrossRefGoogle Scholar
  10. 10.
    Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27CrossRefzbMATHGoogle Scholar
  11. 11.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 886–893Google Scholar
  12. 12.
    Daras P, Axenopoulos A (2010) A 3d shape retrieval framework supporting multimodal queries. Int J Comput Vis 89(2):229–247CrossRefGoogle Scholar
  13. 13.
    Davis JV, Kulis B, Jain P, Sra S, Dhillon IS (2007) Information-theoretic metric learn. In: Machine Learning Proceedings of the Twenty-Fourth International Conference, pp 209–216Google Scholar
  14. 14.
    Duchenne O, Bach F, In SK, Ponce J (2011) A tensor-based algorithm for high-order graph matching. IEEE Trans Pattern Anal Mach Intell 33(12):2383–95CrossRefGoogle Scholar
  15. 15.
    Fang Y, Xie J, Dai G, Wang M, Zhu F, Xu T, Wong E (2015) 3d deep shape descriptor. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2319–2328Google Scholar
  16. 16.
    Funkhouser T, Min P, Kazhdan M, Chen J, Halderman A, Dobkin D, Jacobs D (2003) A search engine for 3d models. Acm Trans Graph 22(1):83–105CrossRefGoogle Scholar
  17. 17.
    Gao Y, Wang M, Tao D, Ji R, Dai Q (2012) 3-d object retrieval and recognition with hypergraph analysis. IEEE Trans Image Process Publ IEEE Signal Process Soc 21(9):4290–303MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Gao Y, Dai Q, Wang M, Zhang N (2011) 3d model retrieval using weighted bipartite graph matching. Signal Process Image Commun 26(1):39–47CrossRefGoogle Scholar
  19. 19.
    Gao Y, Dai Q, Zhang NY (2010) 3d model comparison using spatial structure circular descriptor. Pattern Recogn 43(3):1142–1151CrossRefzbMATHGoogle Scholar
  20. 20.
    Gao Y, Tang J, Hong R, Yan S (2012) Camera constraint-free view-based 3-d object retrieval. IEEE Trans Image Process Publ IEEE Signal Process Soc 21(4):2269–2281MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Gao Z, Zhang H, Xu GP, Xue YB (2015) Multi-perspective and multi-modality joint representation and recognition model for 3d action recognition. Neurocomputing 151:554–564CrossRefGoogle Scholar
  22. 22.
    Gao Z, Nie W, Liu A, Zhang H (2016) Evaluation of local spatial–temporal features for cross-view action recognition. Neurocomputing 173:110–117CrossRefGoogle Scholar
  23. 23.
    Gao Z, Zhang H, Xu GP, Xue YB, Hauptmann AG (2015) Multi-view discriminative and structured dictionary learning with group sparsity for human action recognition. Signal Process 112:83–97CrossRefGoogle Scholar
  24. 24.
    Gao Z, Zhang L-f, Chen M-y, Hauptmann A, Zhang H, Cai A-N (2014) Enhanced and hierarchical structure algorithm for data imbalance problem in semantic extraction under massive video dataset. Multimed Tools Appl 68(3):641–657CrossRefGoogle Scholar
  25. 25.
    Goldberger J, Roweis ST, Hinton GE, Salakhutdinov R (2004) Neighbourhood components analysis. Adv Neural Inf Process Syst 83(6):513–520Google Scholar
  26. 26.
    Guétat G, Maitre M, Joly L, Lai SL, Lee T, Shinagawa Y (2006) Automatic 3-d grayscale volume matching and shape analysis. IEEE Trans Inf Technol Biomed Publ IEEE Eng Med Biol Soc 10(2):362–76CrossRefGoogle Scholar
  27. 27.
    Hsieh CT, Han CC, Shih JL, Lee CH (2015) 3d model retrieval using multiple features and manifold ranking. In: International conference on ubi-media computingGoogle Scholar
  28. 28.
    Ip CY, Lapadat D, Sieger L, Regli WC (2002) Using shape distributions to compare solid models. In: ACM symposium on solid modeling and applications, pp 273–280Google Scholar
  29. 29.
    Kim WY, Kim YS (2000) A region-based shape descriptor using zernike moments. Signal Process Image Commun 16(1-2):95–102CrossRefGoogle Scholar
  30. 30.
    Leibe B, Schiele B (2003) Analyzing appearance and contour based methods for object categorization. In: 2003. Proceedings. 2003 IEEE computer society conference on computer vision and pattern recognition, pp 409–415Google Scholar
  31. 31.
    Mohamed W, Ben Hamza A (2012) Reeb graph path dissimilarity for 3d object matching and retrieval. Vis Comput 28(3):305–318CrossRefGoogle Scholar
  32. 32.
    Nie WZ, An AL, Gao Z, Su YT (2015) Clique-graph matching by preserving global and local structure, pp 4503–4510Google Scholar
  33. 33.
    Nie W, Cao Q, Liu A, Su Y (2017) Convolutional deep learning for 3D object retrieval. Multimedia Syst 23(3):325–332Google Scholar
  34. 34.
    Nie W, Li X, Liu A, Su Y (2017) 3D object retrieval based on Spatial+LDA model. Multimed Tools Appl 76(3):4091–410Google Scholar
  35. 35.
    Novatnack J, Nishino K (2007) Scale-dependent 3d geometric features, pp 1–8Google Scholar
  36. 36.
    Ohbuchi R, Osada K, Furuya T, Banno T (2008) Salient local visual features for shape-based 3d model retrieval. In: IEEE international conference on shape modeling and applications, pp 93–102Google Scholar
  37. 37.
    Ohbuchi R, Furuya T (2009) Scale-weighted dense bag of visual features for 3d model retrieval from a partial view 3d model. In: IEEE International Conference on Computer Vision Workshops, pp 63–70Google Scholar
  38. 38.
    Osada R, Funkhouser T, Chazelle B, Dobkin D (2001) 3d models with shape distributions. In: SMI 2001 International Conference on Shape Modeling and Applications, pp 0154–0154Google Scholar
  39. 39.
    Osada R, Funkhouser T, Chazelle B, Dobkin D (2002) Shape distributions. Acm Trans Graph 21(4):807–832MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Papoiu AD, Emerson NM, Patel TS, Kraft RA, Valdes-Rodriguez R, Nattkemper LA, Coghill RC, Yosipovitch G (2014) Voxel-based morphometry and arterial spin labeling fmri reveal neuropathic and neuroplastic features of brain processing of itch in end-stage renal disease. J Neurophysiol 112(7):1729–38CrossRefGoogle Scholar
  41. 41.
    Paquet E, Rioux M, Murching A, Naveen T, Tabatabai A (2000) Description of shape information for 2-d and 3-d objects. Signal Process Image Commun 16(s 1–2):103–122CrossRefGoogle Scholar
  42. 42.
    Persoon E, Fu KS (1986) Shape discrimination using fourier descriptors. IEEE Trans Syst Cybern 7(3):170–179MathSciNetCrossRefGoogle Scholar
  43. 43.
    Polewski P, Yao W, Heurich M, Krzystek P, Stilla U (2015) Active learning approach to detecting standing dead trees from ALS point clouds combined with aerial infrared imagery. CVPRWorkshop, pp 10–18Google Scholar
  44. 44.
    Regli WC, Cicirello VA (2000) Managing digital libraries for computer-aided design. Comput-Aided Des 32(2):119–132CrossRefGoogle Scholar
  45. 45.
    Sharma A, Jacobs DW (2011) Bypassing synthesis: Pls for face recognition with pose, low-resolution and sketch. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 593–600Google Scholar
  46. 46.
    Shih JL, Lee CH, Wang JT (2007) A new 3d model retrieval approach based on the elevation descriptor. Pattern Recogn 40(1):283–295CrossRefzbMATHGoogle Scholar
  47. 47.
    Shinagawa Y, Kunii TL (1991) Constructing a reeb graph automatically from cross sections. IEEE Comput Graph Appl 11(6):44–51CrossRefGoogle Scholar
  48. 48.
    Sundar H, Silver D, Gagvani N, Dickinson S (2003) Skeleton based shape matching and retrieval. In: Shape modeling international, p 130Google Scholar
  49. 49.
    Tangelder JWH, Veltkamp RC (2003) Polyhedral model retrieval using weighted point sets. In: Shape modeling international, pp 209–229Google Scholar
  50. 50.
    Vavilov D, Dovzhenko D, Anisimov A (2010) Perspectives of stereo 3d tv applications development. In: Software engineering conference, pp 175–178Google Scholar
  51. 51.
    Vinyals O, Toshev A, Bengio S, Erhan D (2015) Show and tell: A neural image caption generator. In: Computer vision and pattern recognition, pp 3156–3164Google Scholar
  52. 52.
    Wang Y, Liu Z, Pang F, Li H (2015) Boosting 3d model retrieval with class vocabularies and distance vector revision, pp 1–5Google Scholar
  53. 53.
    Yeh JS, Chen DY, Chen BY, Ouhyoung M (2005) A web-based three-dimensional protein retrieval system by matching visual similarity. Bioinformatics 21(13):3056–3057CrossRefGoogle Scholar
  54. 54.
    Zhao S, Yao H, Zhang Y, Wang Y, Liu S (2015) View-based 3d object retrieval via multi-modal graph learning. Signal Process 112(C):110–118CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.School of Electronic Information EngineeringTianjin UniversityTianjinChina

Personalised recommendations