Benchmarking for Content-Based Visual Information Search

  • C. H. C. Leung
  • H. H. S. Ip
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1929)


The importance of the visual information search problem has given rise to a large number of systems and prototypes being built to perform such search. While different systems clearly have their particular strengths, they tend to use different collections to highlight the advantages of their algorithms. Consequently, a degree of bias may exist, and it also makes it difficult to make comparisons concerning the relative superiority of different algorithms. In order for the field of visual information search to make further progress, a need therefore exists for a standardised benchmark suite to be developed. By having a uniform measure of search performance, research progress can be more easily recognised and charted, and the resultant synergy will be essential to further development of the field. This paper presents concrete proposals concerning the development of such a benchmark, and by adopting an extensible framework, it is able to cater for a wide variety of applications paradigms and to lend itself to incremental refinement.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    D. Adjeroh, and K. C. Nwosu, “Multimedia database management — requirements and issues”, IEEE Multimedia, Vol. 4, No. 4, 1997, pp. 24–33.CrossRefGoogle Scholar
  2. 3.
    M. Campanai, Del Bimbo, A., and Nesi, P., “Using 3D spatial relationships for image retrieval by contents”, Proc. IEEE Workshop on Visual Languages, 1992.Google Scholar
  3. 4.
    F. Chang, et. al., “Visual information retrieval from large distributed online repositories”, Comm. ACM, Vol. 40, No. 12, 1997, pp. 63–71.CrossRefGoogle Scholar
  4. 5.
    C. Colombo, A. Del Bimbo, and P. Pala, “Semantics in visual information retrieval”, IEEE Multimedia, July-September, 1999.Google Scholar
  5. 7.
    U. Gargi, and R. Kasturi, “Image database querying using a multi-scale localised colour representation”, in Proc. IEEE Workshop on Content-based Access of Image and Video Libraries, Fort Collins, 1999.Google Scholar
  6. 8.
    M. Flickner, et. al., “Query by image and video content: the QBIC system”, IEEE Computer, Vol. 28, No. 9, September 1995, pp. 23–32.Google Scholar
  7. 9.
    J. Gray, “A measure of transaction processing power”, Datamation, 1985.Google Scholar
  8. 10.
    W. I. Grosky, “Managing multimedia information in database systems”, Comm. ACM, Vol. 40, No. 12, 1997, pp. 72–80.CrossRefGoogle Scholar
  9. 11.
    V. N. Gudivada and V. Raghavan, “Modeling and retrieving images by content”, Information Processing & Management, Vol. 33, No. 4, pp. 427–452, 1997.CrossRefGoogle Scholar
  10. 12.
    A. Gupta and R. Jain, “Visual information retrieval”, Comm. ACM, Vol. 40, No.5, 1997, pp. 70–79.CrossRefGoogle Scholar
  11. 13.
    A. Gupta, S. Santini, and R. Jain, “In search of information in visual media”, Comm. ACM, Vol. 40, No.12, 1997, pp. 35–42.CrossRefGoogle Scholar
  12. 14.
    H. H. S. Ip, K. C. K. Law, and S. L. Chan, “An open framework for a multimedia medical document system”, J. Microcomputer Applications, Vol. 18, pp. 215–232, 1995.CrossRefGoogle Scholar
  13. 15.
    K. C. K. Law, H. H. S. Ip,, and F. Wei, “An abstract layered model for hypermedia document system”, Proc. IEEE International Conference on Multimedia Computing and Systems, Florence, 1999, pp. 11–15.Google Scholar
  14. 16.
    G. Lawton, “Storage technology takes center stage”, IEEE Multimedia, Vol. 32, No. 11, 1999, pp. 10–12.MathSciNetGoogle Scholar
  15. 17.
    C. H. C. Leung and D. Hibler, Architecture of a Pictorial Database Management System. British Library Research Report, London, 1991.Google Scholar
  16. 18.
    C. H. C. Leung and D. Sutanto, “Multimedia data modeling and management for semantic content retrieval”, in Handbook of Multimedia Computing, B. Fuhrt (Ed.), CRC Press, 1999.Google Scholar
  17. 19.
    C. H. C. Leung and Z. J. Zheng, “Image data modelling for efficient content indexing”, Proc. IEEE International Workshop on Multi-media Database Management Systems, New York, August 1995, IEEE Computer Society Press, pp. 143–150Google Scholar
  18. 20.
    C. H. C. Leung (Ed.) Visual Information Systems. Springer-Verlag Lecture Notes in Computer Science LNCS 1306, Heidelberg, 1997.Google Scholar
  19. 21.
    C. H. C. Leung, and A. M. Tam, “Validation experiment for structured annotation DS”, ISO/IEC JTC1/SC29/WG11 MPEG00/M5855, Noordwijkerhout, NL, March 2000.Google Scholar
  20. 22.
    C. H. C. Leung, and A. M. Tam, “Workplan for CE on DSs for linguistic structure”, ISO/IEC JTC1/SC29/WG11 MPEG00/W3254, Noordwijkerhout, NL, March 2000.Google Scholar
  21. 23.
    W. S. Li et. al. “Hierarchical image modelling for object-based media retrieval”, Data and Knowledge Engineering 27, 1998, pp. 139–176.zbMATHCrossRefGoogle Scholar
  22. 25.
    D. Narasimhalu, M. Kankanhalli, and J. K. Wu, “Benchmarking multimedia databases”, Multimedia Tools and Applications, Vol. 4, 1997, pp. 333–356.CrossRefGoogle Scholar
  23. 26.
    F. Pereira, “MPEG-7: A standard for content-based audiovisual description”, Proc. 2nd International Conference on Visual Information Systems, San Diego, Dec 1997, pp. 1–4.Google Scholar
  24. 27.
    Salton, G. Automatic Text Processing. Addison-Wesley, Reading, MA,1989.Google Scholar
  25. 28.
    J. Smith and S. F. Chang, “Visually searching the Web for content”, IEEE Multimedia, Vol. 4, No. 3, 1997, pp. 12–20.CrossRefGoogle Scholar
  26. 29.
    W.W.S. So, C. H. C. Leung and Z. J. Zheng, “Analysis and evaluation of search efficiency for image databases”, in Image Databases and Multi-media Search A. Smeulders and R. Jain (Eds.), World Scientific, 1997.Google Scholar
  27. 30.
    R. K. Srihari and Z. Zhang, “A multimedia image annotation, indexing, and retrieval system”, Proc. 1998 ACM SIGIR International Workshop on Multimedia Indexing and Retrieval, Melbourne, August 1998, pp. 29–45.Google Scholar
  28. 31.
    Standard Performance Evaluation Corporation,
  29. 32.
    A. M. Tam and C. H. C. Leung, “A multiple media approach to visual information search,” Proc. 1998 ACM SIGIR International Workshop on Multimedia Indexing and Retrieval, Melbourne, August 1998, pp. 1–6.Google Scholar
  30. 33.
    A. M. Tam, and C. H. C. Leung, “Semantic content retrieval and structured annotation: beyond keywords”, ISO/IEC JTC1/SC29/WG11 MPEG00/M5738, Noordwijkerhout, NL, March 2000.Google Scholar
  31. 34.
    Transaction Processing Performance Council,
  32. 37.
    L. Yang, and Wu, J., “Towards a semantic image database system,” Data & Knowledge Engineering, Vol. 22, No. 3, May 1997, pp. 207–227.zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • C. H. C. Leung
    • 1
  • H. H. S. Ip
    • 2
  1. 1.School of Communications & InformaticsVictoria University of TechnologyVICAustralia
  2. 2.Department of Computer ScienceCity University of Hong KongKowloonHong Kong

Personalised recommendations