Skip to main content
Log in

MUSE: A Content-Based Image Search and Retrieval System Using Relevance Feedback

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The field of Content-Based Visual Information Retrieval (CBVIR) has experienced tremendous growth in the recent years and many research groups are currently working on solutions to the problem of finding a desired image or video clip in a huge archive without resorting to metadata. This paper describes the ongoing development of a CBVIR system for image search and retrieval with relevance feedback capabilities. It supports browsing, query-by-example, and two different relevance feedback modes that allow users to refine their queries by indicating which images are good or bad at each iteration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. S. Aksoy and R.M. Haralick, "Textural features for image database retrieval," in Proc. IEEE Workshop on Content-Based Access of Image and Video Libraries, 1998, pp. 45-49.

  2. S. Aksoy and R.M. Haralick, "Graph-theoretic clustering for image grouping and retrieval," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1998.

  3. J. Ashley et al., "Automatic and semi-automatic methods for image annotation and retrieval in QBIC," in Proc. SPIE Storage and Retrieval for Image and Video Databases, San Diego/La Jolla, CA, USA, 1995.

  4. A. Benitez, M. Beigi, and S.-F. Chang, "Using relevance feedback in content-based image search," IEEE Internet Computing, Vol. 2, No. 4, pp. 59-69, 1998.

    Google Scholar 

  5. M. Bouet and C. Djeraba, "Visual content based retrieval in an image database with relevant feedback," in Proceedings of the International Workshop on Multi-Media Database Management Systems, 1998, pp. 98-105.

  6. R. Brunelli and O. Mich, "Image retrieval by examples," IEEE Transactions on Multimedia, Vol. 2, No. 3, pp. 164-171, 2000.

    Google Scholar 

  7. E.Y. Chang, Beitao Li, and Chen Li, "Toward perception-based image retrieval," in Proceeding IEEE Workshop on Content-based Access of Image and Video Libraries, 2000, pp. 101-105.

  8. S.-F. Chang, A. Eleftheriadis, and R. McClintock, "Next-generation content representation, creation and searching for new media applications in education," in Proceedings of the IEEE, Vol. 86, No. 5, pp. 884-904, 1998.

    Google Scholar 

  9. S.-F. Chang, J.R. Smith, M. Beigi, and A. Benitez, "Visual information retrieval from large distributed online repositories," Communications of the ACM, Vol. 40, No. 12, pp. 63-71, 1997.

    Google Scholar 

  10. T.-S. Chua, C.-X. Chu, and M. Kankanhalli, "Relevance feedback techniques for image retrieval using multiple attributes," in Proceedings of the International Conference on Multimedia Computing and Systems, Florence, Italy, 1999.

  11. G. Ciocca, I. Gagliardi, and R. Schettini, "Quicklook: A content-based image retrieval system with learning capabilities," in Proceeding of the International Conference on Multimedia Computing and Systems, Florence, Italy, 1999.

  12. I.J. Cox, M.L. Miller, T.P. Minka, T.V. Papathomas, and P.N. Yianilos, "The Bayesian image retrieval system, PicHunter: Theory, implementation, and psychophysical experiments," IEEE Transactions on Image Processing, Vol. 9, No. 1, pp. 20-37, 2000.

    Google Scholar 

  13. I. Cox, M. Miller, T. Minka, and P. Yianilos, "An optimized interaction strategy for Bayesian relevance feedback," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1998, pp. 553-558.

  14. I.J. Cox, M.L. Miller, S.M. Omohundro, and P.N. Yianilos, "PicHunter: Bayesian relevance feedback for image retrieval," in Proceedings of the International Conference on Pattern Recognition, Vienna, Austria, 1996.

  15. I.J. Cox, M.L. Miller, S.M. Omohundro, and P.N. Yianilos, "Target testing and the PicHunter Bayesian multimedia retrieval system," in Advanced Digital Libraries ADL'96 Forum, Washington D.C., 1996.

  16. I.J. Cox, M.L. Miller, T. Papathomas, J. Ghosn, and P.N. Yianilos, "Hidden annotation in content based image retrieval," in Proceedings of the IEEE Workshop on Content-Based Access of Image and Video Libraries, 1997.

  17. A. Del Bimbo, "A perspective view on visual information retrieval systems," in Proceedings of the IEEE Workshop on Content-Based Access of Image and Video Libraries, Santa Barbara, California, 1998.

  18. A. Del Bimbo, Visual Information Retrieval, Morgan Kaufmann: San Francisco, CA, 1999.

    Google Scholar 

  19. Y. Deng and B.S. Manjunath, "An efficient low-dimensional color indexing scheme for region-based image retrieval," in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, March 1999, Vol. 6, pp. 3017-3020.

    Google Scholar 

  20. A.D. Doulamis, Y.S. Avrithis, N.D. Doulamis, and S.D. Kollias, "Interactive content-based retrieval in video database using fuzzy classification and relevance feedback," in Proceedings of the International Conference on Multimedia Computing and Systems, Florence, Italy, 1999.

  21. M. Flickner et al., "Query by image and video content: The QBIC system," in Intelligent Multimedia Information Retrieval, M.T. Maybury (Ed.), American Association for Artificial Intelligence (AAAI): Menlo Park, CA, 1997, Ch.1, pp. 7-22.

    Google Scholar 

  22. T. Gevers and A.W.M. Smeulders, "The PicToSeek WWW image search system," in Proceedings of the International Conference on Multimedia Computing and Systems, Florence, Italy, 1999.

  23. T. Gevers and A.W.M. Smeulders, "PicToSeek: Combining color and shape invariant features for image retrieval," IEEE Transactions on Image Processing, Vol. 9, No. 1, pp. 102-119, 2000.

    Google Scholar 

  24. Yihong Gong, G. Proietti, and C. Faloutsos, "Image indexing and retrieval based on human perceptual color clustering," in Proceedings in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1998, pp. 578-583.

  25. A. Gupta and R. Jain, "Visual information retrieval," Communications of the ACM, Vol. 40, No. 5, pp. 71-79, 1997.

    Google Scholar 

  26. J. Han and M. Kamber, Data Mining: Concepts and Techniques, Morgan Kaufmann: San Francisco, 2001.

    Google Scholar 

  27. K. Hirata, S. Mukherjea,W.-S. Li, and Y. Hara, "Integrating image matching and classification for multimedia retrieval on the web," in Proceedings of the International Conference on Multimedia Computing and Systems, Florence, Italy, 1999.

  28. J. Huang, S. Kumar, M. Mitra, W.-J. Zhu, and R. Zabih, "Image indexing using color correlograms," in Proc. IEEE International Conference on Computer Vision and Pattern Recognition, 1997.

  29. L. Kaufman and P.J. Rousseeuw, Finding Groups in Data: An Introduction to Cluster Analysis, Wiley: New York, 1990.

    Google Scholar 

  30. S. Krishnamachari and M. Abdel-Mottaleb, "Image browsing using hierarchical clustering," in Proceedings IEEE International Symposium on Computers and Communications, 1999, pp. 301-307.

  31. A. Kuchinsky et al., "Fotofile: A consumer multimedia organization and retrieval system," in Proceeding of the CHI 99 Conference on Human Factors in Computing Systems: The CHI is the Limit, Pittsburgh, PA, USA, 1999.

  32. E. La Cascia and M. Ardizzone, "Jacob: Just a content-based query system for video database," in Proceedings IEEE International Conference on Acoustics, Speech, and Signal Processing, 1996, Vol. 2, pp. 1216-1219.

    Google Scholar 

  33. C.H.C. Leung and H.H.S. Ip, Benchmarking for content-based visual information search.

  34. W.Y. Ma, "NETRA: A Toolbox for Navigating Large Image Databases," Ph.D. thesis, University of California at Santa Barbara, June 1997.

  35. W.Y. Ma and B.S. Manjunath, "Netra: A toolbox for navigating large image databases," in Proceedings of the IEEE International Conference on Image Processing, Oct. 1997, Vol. 1, pp. 568-571.

    Google Scholar 

  36. O. Marques and B. Furht, "Issues in designing contemporary video database systems," in Proceedings of the IASTED Conference on Internet and Multimedia Systems and Applications, Nassau, Bahamas, 1999.

  37. C. Meilhac and C. Nastar, "Relevance feedback and category search in image databases," in Proceedings of the International Conference on Multimedia Computing and Systems, Florence, Italy, 1999.

  38. T. Minka, "An image database browser that learns from user interaction," MEng thesis, MIT, 1996.

  39. W. Niblack et al., "The QBIC project: Querying images by content using color, texture and shape," in Proc. SPIE Storage and Retrieval for Image and Video Databases, pp. 173-187, 1993.

  40. V.E. Ogle and M. Stonebraker, "Chabot: Retrieval from a relational database of images," IEEE Computer, Vol. 28, No. 9, pp. 40-48, 1995.

    Google Scholar 

  41. M. Ortega, Y. Rui, K. Chakrabarti, S. Mehrotra, and T.S. Huang, "Supporting similarity queries in MARS," in Proc. of ACM Multimedia 1997, pp. 403-413.

  42. A. Pentland, R.W. Picard, and S. Sclaroff, "Photobook: Content-based manipulation of image databases," in Multimedia Tools and Applications, B. Furht (Ed.), Kluwer Academic Publishers: Boston, MA, 1996, Ch. 2.

    Google Scholar 

  43. D. Petkovic, "Challenges and opportunities in search and retrieval for media database," in Proceedings of the IEEE Workshop on Content-Based Access of Image and Video Libraries, Santa Barbara, California, 1998.

  44. J.C. Platt, "Autoalbum: Clustering digital photographs using probabistic model merging," in Proc. IEEE Workshop on Content-Based Access of Image and Video Libraries, 2000.

  45. Y. Rui, T.S. Huang, and S.-F. Chang, "Image retrieval: Current techniques, promising directions and open issues," Journal of Visual Communication and Image Representation, Vol. 10, pp. 39-62, 1999.

    Google Scholar 

  46. Y. Rui, T.S. Huang, and S. Mehrotra, "Content-based image retrieval with relevance feedback in MARS," in Proceedings of IEEE Int. Conf. Image Processing, 1997, pp. 815-818.

  47. Y. Rui, T.S. Huang, and S. Mehrotra, "Relevance feedback techniques in interactive content-based image retrieval," in Proc. S&T SPIE Storage and Retrieval of Images/Video Databases VI, EI'98, 1998.

  48. Y. Rui, T.S. Huang, S. Mehrotra, and M. Ortega, "Automatic matching tool selection using relevance feedback in MARS," in Proc. 2nd Int. Conf. Visual Information Systems, 1997, pp. 109-116.

  49. Y. Rui, T.S. Huang, S. Mehrotra, and M. Ortega, "A relevance feedback architecture for content-based multimedia information retrieval systems," in Proceedings of IEEE Workshop on Content-based Access of Image and Video Libraries, 1997, pp. 82-89.

  50. Y. Rui, T.S. Huang, M. Ortega, and S. Mehrotra, "Relevance feedback: A power tool for interactive content-based image retrieval," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 8, No. 5, pp. 644-655, 1998.

    Google Scholar 

  51. G. Sheikholeslami, W. Chang, and A. Zhang, "Semantic clustering and querying on heterogeneous features for visual data," in Proceedings of the 6th ACM International Multimedia Conference, Bristol, England, 1998.

  52. J.R. Smith and S.-F. Chang, "VisualSEEk: A fully automated content-based image query system," in Proc. ACM Multimedia '96, Boston, MA, Nov. 1996.

  53. J.R. Smith and S.-F. Chang, "Querying by color regions using the visualSEEk content-based visual query system," in Intelligent Multimedia Information Retrieval, M.T. Maybury (Ed.), American Association for Artificial Intelligence (AAAI): Menlo Park, CA, 1997, Ch.2, pp. 23-41.

    Google Scholar 

  54. M.J. Swain and D.H. Ballard, "Indexing via color histograms," in Proc. Third International Conference on Computer Vision, pp. 390-393, 1990.

  55. M.J. Swain and D.H. Ballard, "Color indexing," International Journal of Computer Vision, Vol. 7, pp. 11-32, 1991.

    Google Scholar 

  56. N. Vasconcelos and A. Lippman, "Bayesian relevance feedback for content-based image retrieval," in Proc. IEEE Workshop on Content-Based Access of Image and Video Libraries 2000, pp. 63-67.

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Marques, O., Furht, B. MUSE: A Content-Based Image Search and Retrieval System Using Relevance Feedback. Multimedia Tools and Applications 17, 21–50 (2002). https://doi.org/10.1023/A:1014679605305

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1014679605305

Navigation