Multimedia Tools and Applications

, Volume 21, Issue 1, pp 35–54 | Cite as

Alternating Feature Spaces in Relevance Feedback

  • Fang Qian
  • Mingjing Li
  • Hong-Jiang Zhang
  • Wei-Ying Ma
  • Bo Zhang


Image retrieval using relevance feedback can be treated as a two-class learning and classification process. The user-labelled relevant and irrelevant images are regarded as positive and negative training samples, based on which a classifier is trained dynamically. Then the classifier in turn classifies all images in the database. In practice, the number of training samples is very small because the users are often impatient. On the other hand, the positive samples usually are not representative since they are the nearest ones to the query and thus less informative. The insufficiency of training samples both in quantities and varieties constrains the generalization ability of the classifier significantly. In this paper, we propose a novel relevance feedback approach, which aims to collect more representative samples and hence improve the performance of classifier. Image labeling and classifier training are conducted in two complementary image feature spaces. Since the samples distribute differently in two spaces, the positive samples may be more informative in one feature space than in another. The two complementary feature spaces are alternated iteratively during the feedback process. To choose appropriate complementary feature spaces, we present two methods to measure the complementarities between two feature spaces quantitatively. Our experimental result on 10,000 images indicates that the proposed feedback approach significantly improves image retrieval performance.

image retrieval relevance feedback complementary features representative training samples 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    T. Chang and C.-C.J. Kuo, “Texture analysis and classification with tree-structured wavelet transform,” IEEE Transactions on Image Processing, Vol. 2, No. 4, pp. 429–441, 1993.Google Scholar
  2. 2.
    I.J. Cox, T.P. Minka, T.V. Papathomas, and P.N. Yianilos, “The Bayesian image retrieval system, PicHunter: Theory, implementation, and psychophysical experiments,” IEEE Transactions on Image Processing—Special Issue on Digital Libraries, 2000.Google Scholar
  3. 3.
    P. Hong, Q. Tian, and T.S. Huang, “Incorporate support vector machines to content-based Image retrieval with relevance feedback,” in IEEE Int'l. Conf. on Image Processing, Vancouver, Canada, Sept. 10–13, 2000.Google Scholar
  4. 4.
    J. Huang, S.R. Kumar, M. Mitra, W. Zhu, and R. Zabih, “Image indexing using color correlograms,” in Proc. IEEE Conf. Computer Vision Pattern Recognition, 1997, pp. 762–768.Google Scholar
  5. 5.
    Y. Ishikawa and R. Subramanya, “MindReader: Query databases through multiple examples,” in Proc. of the 24th VLDB Conference, New York, 1998.Google Scholar
  6. 6.
    A.K. Jain and A. Vailaya, “Image retrieval using color and shape,” Pattern Recognition, Vol. 9, No. 8, pp. 1233–1244, Aug. 1996.Google Scholar
  7. 7.
    B.S. Manjunath and W.Y. Ma, “Texture features for browsing and retrieval of image data,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 18, No. 8, pp. 837–842, Aug. 1996.Google Scholar
  8. 8.
    J.C. Mao and A.K. Jain, “Texture classification and segmentation using multiresolution simultaneous autoregressive models,” Pattern Recognition, Vol. 25, pp. 173–188, 1992.Google Scholar
  9. 9.
    F. Qian, M. Li, W. Ma, F. Lin, and B. Zhang, “Alternating feature spaces in relevance feedback,” in 3rd International Workshop on Multimedia Information Retrieval, Ottawa, Canada, Oct. 2001.Google Scholar
  10. 10.
    Y. Rui, T.S. Huang, M. Ortega, and S. Mehrotra, “Relevance feedback: A power tool in interactive content-based image retrieval,” IEEE Transaction on Circuits and Systems for Video Technology, Special Issue on Segmentation, Description, and Retrieval of Video Content, Vol. 8, No. 5, pp. 644–655, Sept. 1998.Google Scholar
  11. 11.
    Y. Rui and T.S. Huang, “Optimizing learning in image retrieval,” in Proc. of IEEE Int. Conf. on Computer Vision and Pattern Recognition, Hilton Head, SC, June 2000.Google Scholar
  12. 12.
    M. Stricker and M. Orengo, “Similarity of color images,” in SPIE Proceedings, 1995, Vol. 2420.Google Scholar
  13. 13.
    Z. Su, H.J. Zhang, and S. Ma, “Relevant feedback using a Bayesian classifier in content-based image retrieval,” in SPIE Electronic Imaging, San Jose, CA, Jan. 2001.Google Scholar
  14. 14.
    K. Tieu and P. Viola, “Boosting image retrieval,” in Proc. IEEE Conf. ComputerVision and Pattern Recognition, Hilto Head Island, SC, June 2000.Google Scholar
  15. 15.
    S. Tong and E. Chang, “Support vector machine active learning for image retrieval,” in Proc.ACM Multimedia 2001, Ottawa, Canada, Sept. 2001.Google Scholar
  16. 16.
    N. Vasconcelos and A. Lippman, “Learning from user feedback in image retrieval systems,” in NIPS'99, Denver, Colorado, 1999.Google Scholar
  17. 17.
    Y. Wu, Q. Tian, and T.S. Huang, “Discriminant-EM algorithm with application to image retrieval,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, South Carolina, June 2000.Google Scholar
  18. 18.
    K. Xu, “Texture representations for content-based retrieval,” Master thesis, Electrical and Computer Engineering Department, Rutgers University, May 2000.Google Scholar
  19. 19.
    L. Zhang, F.Z. Lin, and B. Zhang, “Support vector learning for image retrieval,” in IEEE International Conference on Image Processing, pp. 721–724. Thessaloniki, Greece, Oct. 2001.Google Scholar
  20. 20.
    L. Zhang, F.Z. Lin, and B. Zhang, “Aneural network based self-learning algorithm of image retrieval,” Chinese Journal of Software, Vol. 12, No. 10, pp. 1479–1485, Oct. 2001.Google Scholar
  21. 21.
    L. Zhang and B. Zhang, “A geometrical representation of McCulloch-Pitts neural model and its application,” IEEE Transaction on Neural Networks, Vol. 10, No. 4, pp. 925–929, 1999.Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • Fang Qian
    • 1
  • Mingjing Li
    • 2
  • Hong-Jiang Zhang
    • 2
  • Wei-Ying Ma
    • 2
  • Bo Zhang
    • 1
  1. 1.Department of Computer Science and TechnologyTsinghua UniversityBeijingChina
  2. 2.Microsoft Research AsiaBeijingChina

Personalised recommendations