Abstract
Drawings and sketches are a natural way for people to communicate ideas, but it remains challenging to develop automated systems that can robustly recognize and interpret what is drawn. Most commonly, a drawing is first processed to obtain a low-level representation of that drawing in terms of lines or strokes, and this representation is then searched for matches to known object templates. In this chapter we propose two template-based methods for sketch recognition. A novel feature of these methods is that they both employ a hierarchy-of-parts template model that provides explicit support for templates with optional parts. This captures significant parts-based variation which would otherwise require a multitude of fixed-structure templates to model. The first method is developed for recognition in drawings consisting of sets of connected strokes and is applied as an interface for creating 3D models of airplanes, mugs, and fish. The second method allows for the recognition of more unstructured objects such as faces, plants, and sailboats in drawings that may also contain disjoint strokes. Neither method relies on the timing information of the input strokes, which may not be available for photographed or scanned drawings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Stroke segmentation is turned off when drawing the template graph.
- 2.
We use w 1=2,w 2=1,w 3=1.
- 3.
We use σ 1=22°,σ 2=0.25,σ 3=0.1.
References
Alvarado, C., Davis, R.: Sketchread: a multi-domain sketch recognition engine. In: UIST ’04 ACM Symposium on User Interface Software and Technology, pp. 23–32 (2004)
Belongie, S., Malik, J.: Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(24), 509–522 (2002)
Crandall, D., Felzenszwalb, P.F., Huttenlocher, D.P.: Spatial priors for part-based recognition using statistical models. In: Proceedings of CVPR, pp. 10–17 (2005)
Felzenszwalb, P.F., Huttenlocher, D.P.: Pictorial structures for object recognition. International Journal of Computer Vision 61(1), 55–79 (2005)
Fischler, M.A., Elschlager, R.A.: The representation and matching of pictorial structures. IEEE Transactions on Computers 22(1) (1973)
Kaelbling, L.P., Lozano-Perez, T.: Learning three-dimensional shape models for sketch recognition. Technical Report, MIT CSAIL, January 2005
Kara, L.B., Stahovich, T.F.: Hierarchical parsing and recognition of hand-sketched diagrams. In: Proceedings of UIST’04 (2004)
Mackenzie, G., Alechina, N.: Classifying sketches of animals using an agent-based system. In: Proceedings of the 10th International Conference CAIP. Springer Lecture Notes in Computer Science, vol. 2756, pp. 521–529. Springer, Berlin (2003)
Mahoney, J.V., Fromherz, M.P.J.: Three main concerns in sketch recognition and an approach to addressing them. In: AAAI Spring Symposium on Sketch Understanding (2002)
Oltmans, M., Alvarado, C., Davis, R.: Etcha sketches: Lessons learned from collecting sketch data. In: Making Pen-Based Interaction Intelligent and Natural. AAAI Fall Symposium (2004)
Qi, Y., Szummer, M., Minka, T.P.: Diagram structure recognition by Bayesian conditional random fields. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) (2005)
Rubine, D.: Specifying gestures by example. In: SIGGRAPH ’91, pp. 329–337 (1991)
Saund, E., Mahoney, J., Fleet, D., Larner, D., Lank, E.: Perceptual organization as a foundation for intelligent sketch editing. In: AAAI Spring Symposium on Sketch Understanding (2002)
Sezgin, T.M., Davis, R.: HMM-based efficient sketch recognition. In: Proceedings of the International Conferences on Intelligent User Interfaces (IUI’05), pp. 281–283, 9–12 January 2005. ACM Press, New York (2005)
Shilman, M., Viola, P.: Spatial recognition and grouping of text and graphics. In: Eurographics Workshop on Sketch-Based Interfaces and Modeling (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag London Limited
About this chapter
Cite this chapter
van de Panne, M., Sharon, D. (2011). Flexible Parts-based Sketch Recognition. In: Jorge, J., Samavati, F. (eds) Sketch-based Interfaces and Modeling. Springer, London. https://doi.org/10.1007/978-1-84882-812-4_6
Download citation
DOI: https://doi.org/10.1007/978-1-84882-812-4_6
Publisher Name: Springer, London
Print ISBN: 978-1-84882-811-7
Online ISBN: 978-1-84882-812-4
eBook Packages: Computer ScienceComputer Science (R0)