Graphics-Based Retrieval of Color Image Databases Using Hand-Drawn Query Sketches

  • Gerhard Rigoll
  • Stefan Müller
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1941)


This paper presents a novel approach to graphics-based information retrieval validated with an experimental system that is able to perform integrated shape and color based image retrieval with hand-drawn sketches which can be presented in rotation-, scale-, and translation-invariant mode. Due to the use of Hidden Markov Models (HMMs), an elastic matching of shapes can be performed, which allows the retrieval of shapes by applying simple sketches. Since these sketches represent hand-made line drawings and can be augmented with color features, the resulting user query represents a complex graphics structure that has to be analyzed for retrieving the image database. The database elements (mostly images of hand tools) are represented by HMMs which have been modified in order to achieve the desired rotation invariance property. Invariance with respect to scaling and translation is achieved by the feature extraction, which is a polar sampling technique, with the center of the sampling raster positioned at the shapes’s center of gravity. The outcome of the feature extraction step is also known as a shape matrix, which is a shape descriptor that has been already used occasionally in image processing tasks. The image retrieval system showed good retrieval results even with unexperienced users, which is demonstrated by a number of query sketches and corresponding retrieval images in this paper.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. Goshtasby. Description and Discrimination of Planar Shapes Using Shape Matrices. IEEE Trans. on PAMI, 7(6):738–743, Nov. 1985. 257, 259Google Scholar
  2. [2]
    V. N. Gupta, M. Lenning, and P. Mermelstein. Integration of Acoustic Information in a Large Vocabulary Word Recognizer. In Proc. IEEE ICASSP, pages 697–700, Dallas, 1997. 261Google Scholar
  3. [3]
    A. K. Jain and A. Vailaya. Image Retrieval Using Color and Shape. Pattern Recognition, 29(8):1233–1244, 1996. 258CrossRefGoogle Scholar
  4. [4]
    S. Kuo and O. Agazzi. Keyword Spotting in Poorly Printed Documents Using Pseudo 2-D Hidden Markov Models. IEEE Trans. on PAMI, 13(11):1172–1184, Nov. 1991. 263Google Scholar
  5. [5]
    B. M. Mehtre, M. S. Kankanhalli, and W. F. Lee. Shape Measures for Content Based Image Retrieval: A Comparison. Pattern Recognition, 33(3):319–337, 1997. 256, 263Google Scholar
  6. [6]
    S. Müller, S. Eickeler, and G. Rigoll. Image Database Retrieval of Rotated Objects by User Sketch. In Proc. IEEE Workshop on CBAIVL, pages 40–44, Santa Barbara, 1998. 257, 258, 260, 261, 264Google Scholar
  7. [7]
    L. R. Rabiner and B. H. Juang. An Introduction to Hidden Markov Models. IEEE ASSP Magazine, pages 4–16, 1986. 260Google Scholar
  8. [8]
    R. Sabourin, J.-P. Drouhard, and E. S. Wah. Shape Matrices as a Mixed Shape Factor for O.-line Signature Verification. In Proc. Intern. Conference on Document Analysis and Recognition (ICDAR), pages 661–665, Ulm (Germany), 1997. 257Google Scholar
  9. [9]
    A. Taza and C. Y. Suen. Discrimination of Planar Shapes Using Shape Matrices. IEEE Trans. on SMC, 19(5):1281–1289, Sep/Oct 1989. 257, 258, 259, 261, 262Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Gerhard Rigoll
    • 1
  • Stefan Müller
    • 1
  1. 1.Department of Computer Science, Faculty of Electrical EngineeringGerhard-Mercator-University DuisburgDuisburgGermany

Personalised recommendations