Skip to main content
SpringerLink
Log in

Improving retrieval performance by region constraints and relevance feedback

  • Pattern Recognition and Image Processing
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In this paper, region features and relevance feedback are used to improve the performance of CBIR. Unlike existing region-based approaches where either individual regions are used or only simple spatial layout is modeled, the proposed approach simultaneously models both region properties and their spatial relationships in a probabilistic framework. Furthermore, the retrieval performance is improved by an adaptive filter based relevance feedback. To illustrate the performance of the proposed approach, extensive experiments have been carried out on a large heterogeneous image collection with 17,000 images, which render promising results on a wide variety of queries.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Rui, Y, Huang T S, Chang S F. Image retrieval: Current techniques, promising directions and open issues.Journal of Visual Communication and Image Representation, 1999, 10: 39–62.

    Article  Google Scholar 

  2. Smeulders A W M, Worring M, Santini S. Content-based image retrieval at the end of the early years.IEEE Trans. PAMI, 2000, 22(12): 1349–1380.

    Google Scholar 

  3. Niblack Wet al. The QBIC project: Querying images by content using color texture and shape. InSPIE Proc. Storage and Retrieval for Image and Video Databases, San Jose, 1993, 1908: 173–187.

  4. Pentland A, Picard R, Sclaroff S. Photobook: Contentbased manipulation of image databases.Int. J. Comput. Vis., 1996, 18(3): 233–254, 1996.

    Article  Google Scholar 

  5. Smith J R, Chang S F. VisualSEEK: A fully automated content-based image query system.ACM Multimedia, Bostan MA, 1996.

  6. Rui Y, Huang T, Mehrotra S. Content-based image retrieval with relevance feedback in MARS. InProc. IEEE Int. Conf. on Image Processing, 1997, pp.87–98.

  7. Ma W Y, Manjunath B S. Netra: A toolbox for navigation large image database. InProc. IEEE ICIP, 1997, pp.568–571.

  8. Carson C, Belongie S, Greenspan H, Malik J. Regionbased image querying. InProc. IEEE Workshop on Content-Based Access of Image and Video Libraries, 1997, pp.42–49.

  9. Li J, Wang J Z, Wiederhold G. IRM: Integrated region matching for image retrieval. InProc. ACM Multimedia 2000, Los Angeles, CA, 2000, pp.147–156.

  10. Rui Y, Huang T S. Optimizing learning in image retrieval. InProc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR2000), 2000, 1: 236–243.

  11. Tian Q, Wu Y, Huang T S. Combine user defined region-of-interest and spatial layout for image retrieval. InIEEE International Conference on Image Processing, Vancouver, BC, 2000, 3: 746–749.

  12. Gouet V, Boujemaa N. Object-based queries using color points of interest. InProc. IEEE Workshop on Content-Based Access of Images and Videos (CBAIVL), Kauai, Hawaii, 2001, pp.30–36.

  13. Tian Q, Sebe N, Lew M S, Loupias E, Huang T S. Image retrieval using wavelet-based salient points. InJournal of Electronic Imaging, Special Issue on Storage and Retrieval of Digital Media, 2001, 10(4): 835–849.

  14. Kam A H, Ng T T, Kingsbury N G, Fitzgerald W J. Content based image retrieval through object extraction and querying. InProc. IEEE Workshop on Content-Based Access of Image and Video Libraries (CBAIVL), 2000, pp.91–95.

  15. Ko B C, Lee H S, Byun H. Region-based image retrieval system using efficient feature description. InProc. 15th International Conference on Pattern Recognition, 2000, Vol.4, 283–286.

  16. Moghaddam B, Biermann H, Margaritis D. Image retrieval with local and spatial queries. InProc. IEEE ICIP 2000, 2000, vol.2, 542–545.

  17. Wang W, Song Y, Zhang A. Semantics retrieval by content and context of image regions. InProc. the 15th International Conference on Vision Interface, Calgary, Canada, May 27–29, 2002.

  18. Xu Y, Duygulu P, Saber Eet al. Object based image retrieval based on multi-level segmentation. InProc. IEEE ICASSP, Istanbul, Turkey, 2000, Vol.6, 2019–2022.

  19. Deng Y, Manjunath B S. An efficient low-dimensional color indexing scheme for region-based image retrieval. InProc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), 1999, pp.3017–3020.

  20. Smith J R, Chang S F. SaFe: A general framework for integrated spatial and feature image search.IEEE First Workshop on Multimedia Signal Processing, 1997, pp.301–306.

  21. Pfefferkorn C. A heuristic problem solving design system for equipment or furniture layouts.Communications of the ACM, 1975, 18(5): 286–297.

    Article  Google Scholar 

  22. Kazuyoshi H, Fumio M. Constraint-based approach for automatic spatial layout planning. InProc. the 11th Conference on Artificial Intelligence for Applications, Los Angeles, CA, USA, 1995, pp.38–45.

  23. Anil K. Jain. Fundamentals of Digital Image Processing. Prentice-Hall Press, 1989, pp.412–413.

  24. Pavlidis T, Liow Y T. Integrating region growing and edge detection.IEEE Trans. Pattern Analysis and Machine Intelligence, 1990, 12(3):225–233.

    Article  Google Scholar 

  25. Felzenszwalb P F, Huttenlocher D P. Image segmentation using local variation. InProc. IEEE Computer Vision and Pattern Recognition, Santa Barbara, CA, USA, 1998, pp.98–104.

  26. Leu J G. Computing a shape moment from its boundary.Pattern Recognition, 1991, 10: 949–957.

    Article  MathSciNet  Google Scholar 

  27. Gorman J W, Mitchell, O R, Kuhl F P. Partial shape recognition using dynamic programming.IEEE Trans. PAMI, 1988, 10: 257–266.

    Google Scholar 

  28. Cox I J, Miller M L, Minka T Pet al. The Bayesian image retrieval system, Pichunter: Theory, implementation, and psychophysical experiments.IEEE Trans. Image Processing, 2000, 9(3): 524–524.

    Article  Google Scholar 

  29. Zhou X, Hunag T S. Comparing discriminate transformations and SVM for learning during multimedia retrieval.ACM Multimedia'2001, 2001, pp.137–146.

  30. Anil J, Douglas Z. Feature selection: Evaluation, application, and small sample performance.IEEE PAMI, 1997, 19(2): 153–158.

    Google Scholar 

  31. Pudil P, Novovicova J, Kittler J. Floating search methods in feature selection.Pattern Recognition Letters, 1994, 15: 1119–1125.

    Article  Google Scholar 

  32. Wang T, Rui Y, Hu S M. Optimal adaptive learning for image retrieval. InProc. IEEE Computer Vision and Pattern Recognition (CVPR2001), Kauai, Hawaii, December 11–13, 2001, pp.1140–1147.

  33. Rodden K, Basalaj W, Sinclair Det al. Does organization by similarity assist image browsing. InProc. ACM Compute-Human Interaction (CHI), 2001, pp.190–197.

  34. Salton G, McGill M J. Introduction to Modern Information Retrieval. McGraw-Hill Book Company, New York, 1982.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Tsinghua University, 100084, Beijing, P.R. China

    Tao Wang & Jia-Guang Sun

  2. Statistical Computing Group, Intel China Research Center, 100020, Beijing, P.R. China

    Tao Wang

  3. Microsoft Research, One Microsoft Way, 98052, Redmond, WA, USA

    Yong Rui

Authors
  1. Tao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Rui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jia-Guang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Wang.

Additional information

Supported by the National Natural Science Foundation of China under Grant No.69902004 and the National Key Project of China under Grant No.2001BA201A07.

Wang Tao is a Ph.D. candidate in Department of Computer Science and Technology at Tsinghua University. He received the B.S. degree in the University of Science and Technology of China in 1996, the M.S. degree from Chinese Academy of Sciences in 1999. His current research interests are content-based image retrieval, pattern recognition, image processing and computer graphics.

Yong Rui received his Ph.D. degree in electrical and computer engineering in 1999 from the University of Illinois at Urbana-Champaign. Since March 1999, he has been a researcher in Microsoft Research, Redmond. His research interests include multimedia systems, distance learning and distributed meetings, image/video/audio processing, computer vision and machine learning. He has published five book chapters, six journal papers, and over forty refereed conference papers in the above areas. Dr. Rui holds six U.S. pending patents and is a member of ACM and IEEE.

Jia-Guang Sun is a professor in Department of Computer Science and Technology at Tsinghua University. He is also the Director of National CAD Engineering Center at Tsinghua University and an Academician of the Chinese Academy of Engineering. He received the B.S degree in Computer Science from Tsinghua University in 1972. During 1985 to 1986, he was a visiting scholar in UCLA. His current research interests are computer-aided geometric design, computer graphics and product data management.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, T., Rui, Y. & Sun, JG. Improving retrieval performance by region constraints and relevance feedback. J. Comput. Sci. & Technol. 19, 413–422 (2004). https://doi.org/10.1007/BF02944911

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02944911

Keywords

Search

Navigation