Multimedia Tools and Applications

, Volume 65, Issue 3, pp 363–385 | Cite as

Sketch-based 3D model retrieval by incorporating 2D-3D alignment

  • Bo Li
  • Henry Johan


Sketch-based 3D model retrieval is very important for applications such as 3D modeling and recognition. In this paper, a sketch-based retrieval algorithm is proposed based on a 3D model feature named View Context and 2D relative shape context matching. To enhance the accuracy of 2D sketch-3D model correspondence as well as the retrieval performance, we propose to align a 3D model with a query 2D sketch before measuring their distance. First, we efficiently select some candidate views from a set of densely sampled views of the 3D model to align the sketch and the model based on their View Context similarities. Then, we compute the more accurate relative shape context distance between the sketch and every candidate view, and regard the minimum one as the sketch-model distance. To speed up retrieval, we precompute the View Context and relative shape context features of the sample views of all the 3D models in the database. Comparative and evaluative experiments based on hand-drawn and standard line drawing sketches demonstrate the effectiveness and robustness of our approach and it significantly outperforms several latest sketch-based retrieval algorithms.


Sketch-based 3D model retrieval 2D-3D alignment View context 


  1. 1.
    Adamek T, O’Connor NE (2004) A multiscale representation method for nonrigid shapes with a single closed contour. IEEE Trans Circuits Syst Video Technol 14(5):742–753CrossRefGoogle Scholar
  2. 2.
    Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522CrossRefGoogle Scholar
  3. 3.
    Cao L, Liu J, Tang X (2006) 3D object retrieval using 2D line drawing and graph based relevance reedback. In: Proc. of the 14th annual ACM international conference on Multimedia, MULTIMEDIA ’06, pp 105–108Google Scholar
  4. 4.
    Chalechale A, Mertins A, Naghdy G (2004) Edge image description using angular radial partitioning. IEE Proc, Vis Image Signal Process 151(2):28–41CrossRefGoogle Scholar
  5. 5.
    Chen DY, Tian XP, Shen YT, Ouhyoung M (2003) On visual similarity based 3D model retrieval. Comput Graph Forum 22(3):223–232CrossRefGoogle Scholar
  6. 6.
    Cole F, Golovinskiy A, Limpaecher A, Barros HS, Finkelstein A, Funkhouser T, Rusinkiewicz S (2008) Where do people draw lines? ACM Trans Graph 27(3):1–11CrossRefGoogle Scholar
  7. 7.
    Doug D, Adam F, Szymon R, Anthony S (2003) Suggestive contours for conveying shape. ACM Trans Graph (Proc. SIGGRAPH 2003) 22(3):848–855CrossRefGoogle Scholar
  8. 8.
    Fang R, Godil A, Li X, Wagan A (2008) A new shape benchmark for 3D object retrieval. In: Proc. of the 4th international symposium on advances in visual computing (ISVC), pp 381–392Google Scholar
  9. 9.
    Feldmar J, Ayache N, Betting F (1997) 3D-2D projective registration of free-form curves and surfaces. Comput Vis Image Underst 65(3):403–424CrossRefGoogle Scholar
  10. 10.
    Fonseca MJ, Ferreira A, Jorge JA (2004) Towards 3D modeling using sketches and retrieval. In: Sketch-based interfaces and modeling, pp 127–136Google Scholar
  11. 11.
    Funkhouser TA, Min P, Kazhdan MM, Chen J, Halderman JA, Dobkin DP, Jacobs DP (2003) A search engine for 3D models. ACM Trans Graph 22(1):83–105CrossRefGoogle Scholar
  12. 12.
    Hou S, Ramani K (2006) Sketch-based 3D engineering part class browsing and retrieval. In: Proc. of EUROGRAPHICS workshop on sketch-based interfaces and modeling, pp 131–138Google Scholar
  13. 13.
    Hou S, Ramani K (2007) Classifier combination for sketch-based 3D part retrieval. Comput Graph 31(4):598–609CrossRefGoogle Scholar
  14. 14.
    Huttenlocher D, Ullman S (1987) Object recognition using alignment. In: Proc. of IEEE international conference on computer vision (ICCV), pp 102–111Google Scholar
  15. 15.
    Jonker R, Volgenant A (1987) A shortest augmenting path algorithm for dense and sparse linear assignment problems. Computing 38(4):325–340MathSciNetMATHCrossRefGoogle Scholar
  16. 16.
    Kanai S (2008) Content-based 3D mesh model retrieval from hand-written sketch. Int J Interact Des Manuf 2(2):87–98CrossRefGoogle Scholar
  17. 17.
    Kazhdan M, Funkhouser T, Rusinkiewicz S (2003) Rotation invariant spherical harmonic representation of 3D shape descriptors. In: Proc. of the symposium on geometry processing, pp 156–164Google Scholar
  18. 18.
    Lee J, Funkhouser T (2008) Sketch-based search and composition of 3D models. In: Proc. of EUROGRAPHICS workshop on sketch-based interfaces and modeling, pp 97–104Google Scholar
  19. 19.
    Li B, Johan H (2010) View context: a 3D model feature for retrieval. In: Boll S et al (eds) MMM 2010. LNCS, vol 5916. Springer, Heidelberg, pp 185–195Google Scholar
  20. 20.
    Li B, Johan H (2011) View context based 2D sketch-3D model alignment. In: 2011 IEEE workshop on applications of computer vision (WACV), pp 45–50Google Scholar
  21. 21.
    Li B, Johan H (2011) 3D model retrieval using hybrid features and class information. Multimedia tools and applications, pp 1–26 (online first version)Google Scholar
  22. 22.
    Loop C (1987) Smooth subdivision surfaces based on triangles. Master thesis, University of UtahGoogle Scholar
  23. 23.
    Mori G, Belongie S, Malik J (2005) Efficient shape matching using shape contexts. IEEE Trans Pattern Anal Mach Intell 27(11):1832–1837CrossRefGoogle Scholar
  24. 24.
    Napoléon T, Sahbi H (2010) From 2D silhouettes to 3D object retrieval: contributions and benchmarking. EURASIP J Image Video Process 2010:1–17CrossRefGoogle Scholar
  25. 25.
    Napoléon T, Sahbi H (2010) Sketch-driven mental 3D object retrieval. SPIE, p 75260LGoogle Scholar
  26. 26.
    Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recogn 29(1):51–59CrossRefGoogle Scholar
  27. 27.
    Olsen L, Samavati FF, Sousa MC, Jorge JA (2009) Sketch-based modeling: a survey. Comput Graph 33(1):85–103CrossRefGoogle Scholar
  28. 28.
    Osada R, Funkhouser T, Chazelle B, Dobkin D (2001) Matching 3D models with shape distributions. In: Proc. of shape modeling and applications, pp 154–166Google Scholar
  29. 29.
    Pu J, Ramani K (2006) On visual similarity based 2D drawing retrieval. Comput-aided Des 38(3):249–259CrossRefGoogle Scholar
  30. 30.
    Pu J, Ramani K (2006) ShapeLab: a unified framework for 2D & 3D shape retrieval. In: Proc. of the 3rd international symposium on 3D data processing, visualization and transmission (3DPVT 2006), pp 1072–1079Google Scholar
  31. 31.
    Buss SR (2003) 3D computer graphics: a mathematical introduction with OpenGL. Cambridge University Press, Cambridge, MACrossRefGoogle Scholar
  32. 32.
    Saavedra J, Bustos B, Scherer M, Schreck T (2011) STELA: sketch-based 3D model retrieval using a structure-based local approach. In: Proc. of the first ACM international conference on multimedia retrieval (ICMR’11), p 26Google Scholar
  33. 33.
    Saavedra JM, Bustos B (2010) An improved histogram of edge local orientations for sketch-based image retrieval. In: Goesele M, Roth S, Kuijper A, Schiele B, Schindler K (eds) DAGM-symposium. Lecture notes in computer science, vol 6376. Springer, pp 432–441Google Scholar
  34. 34.
    Shilane P, Min P, Kazhdan M, Funkhouser T (2004) The Princeton shape benchmark. In: Proc. of shape modeling and applications, pp 167–178Google Scholar
  35. 35.
    Snodgrass JG, Vanderwart M (1980) A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. J Exp Psychol Hum Learn Mem 6(2):174–215CrossRefGoogle Scholar
  36. 36.
    Veltkamp RC, ter Haar FB (2007) SHREC 2007 3D retrieval contest. Technical report UU-CS-2007-015, Department of Information and Computing Sciences, Utrecht UniversityGoogle Scholar
  37. 37.
    Vranic DV (2004) 3D model retrieval. PhD thesis, University of LeipzigGoogle Scholar
  38. 38.
    Wang J, He Y, Tian H, Cai H (2008) Retrieving 3D CAD model by freehand sketches for design reuse. Adv Eng Inf 22(3):385–392CrossRefGoogle Scholar
  39. 39.
    Yoon SM, Scherer M, Schreck T, Kuijper A (2010) Sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours. In: ACM multimedia, pp 193–200Google Scholar
  40. 40.
    Zamora S, Sherwood T (2010) Sketch-based recognition system for general articulated skeletal figures. In: Proc. of EUROGRAPHICS workshop on sketch-based interfaces and modeling, pp 119–126Google Scholar
  41. 41.
    Zhang D, Lu G (2002) Shape-based image retrieval using generic Fourier descriptor. Signal Process: Image Commun 17(10):825–848CrossRefGoogle Scholar
  42. 42.
    Zhang D, Luo G (2002) An integrated approach to shape based image retrieval. In: Proc. of the 5th Asian conference on computer vision (ACCV 2002), pp 652–657Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.School of Computer EngineeringNanyang Technological UniversitySingaporeSingapore

Personalised recommendations