Benchmarking for Content-Based Visual Information Search
The importance of the visual information search problem has given rise to a large number of systems and prototypes being built to perform such search. While different systems clearly have their particular strengths, they tend to use different collections to highlight the advantages of their algorithms. Consequently, a degree of bias may exist, and it also makes it difficult to make comparisons concerning the relative superiority of different algorithms. In order for the field of visual information search to make further progress, a need therefore exists for a standardised benchmark suite to be developed. By having a uniform measure of search performance, research progress can be more easily recognised and charted, and the resultant synergy will be essential to further development of the field. This paper presents concrete proposals concerning the development of such a benchmark, and by adopting an extensible framework, it is able to cater for a wide variety of applications paradigms and to lend itself to incremental refinement.
Unable to display preview. Download preview PDF.
- 3.M. Campanai, Del Bimbo, A., and Nesi, P., “Using 3D spatial relationships for image retrieval by contents”, Proc. IEEE Workshop on Visual Languages, 1992.Google Scholar
- 5.C. Colombo, A. Del Bimbo, and P. Pala, “Semantics in visual information retrieval”, IEEE Multimedia, July-September, 1999.Google Scholar
- 7.U. Gargi, and R. Kasturi, “Image database querying using a multi-scale localised colour representation”, in Proc. IEEE Workshop on Content-based Access of Image and Video Libraries, Fort Collins, 1999.Google Scholar
- 8.M. Flickner, et. al., “Query by image and video content: the QBIC system”, IEEE Computer, Vol. 28, No. 9, September 1995, pp. 23–32.Google Scholar
- 9.J. Gray, “A measure of transaction processing power”, Datamation, 1985.Google Scholar
- 15.K. C. K. Law, H. H. S. Ip,, and F. Wei, “An abstract layered model for hypermedia document system”, Proc. IEEE International Conference on Multimedia Computing and Systems, Florence, 1999, pp. 11–15.Google Scholar
- 17.C. H. C. Leung and D. Hibler, Architecture of a Pictorial Database Management System. British Library Research Report, London, 1991.Google Scholar
- 18.C. H. C. Leung and D. Sutanto, “Multimedia data modeling and management for semantic content retrieval”, in Handbook of Multimedia Computing, B. Fuhrt (Ed.), CRC Press, 1999.Google Scholar
- 19.C. H. C. Leung and Z. J. Zheng, “Image data modelling for efficient content indexing”, Proc. IEEE International Workshop on Multi-media Database Management Systems, New York, August 1995, IEEE Computer Society Press, pp. 143–150Google Scholar
- 20.C. H. C. Leung (Ed.) Visual Information Systems. Springer-Verlag Lecture Notes in Computer Science LNCS 1306, Heidelberg, 1997.Google Scholar
- 21.C. H. C. Leung, and A. M. Tam, “Validation experiment for structured annotation DS”, ISO/IEC JTC1/SC29/WG11 MPEG00/M5855, Noordwijkerhout, NL, March 2000.Google Scholar
- 22.C. H. C. Leung, and A. M. Tam, “Workplan for CE on DSs for linguistic structure”, ISO/IEC JTC1/SC29/WG11 MPEG00/W3254, Noordwijkerhout, NL, March 2000.Google Scholar
- 26.F. Pereira, “MPEG-7: A standard for content-based audiovisual description”, Proc. 2nd International Conference on Visual Information Systems, San Diego, Dec 1997, pp. 1–4.Google Scholar
- 27.Salton, G. Automatic Text Processing. Addison-Wesley, Reading, MA,1989.Google Scholar
- 29.W.W.S. So, C. H. C. Leung and Z. J. Zheng, “Analysis and evaluation of search efficiency for image databases”, in Image Databases and Multi-media Search A. Smeulders and R. Jain (Eds.), World Scientific, 1997.Google Scholar
- 30.R. K. Srihari and Z. Zhang, “A multimedia image annotation, indexing, and retrieval system”, Proc. 1998 ACM SIGIR International Workshop on Multimedia Indexing and Retrieval, Melbourne, August 1998, pp. 29–45.Google Scholar
- 31.Standard Performance Evaluation Corporation, http://open.specbench.org/.
- 32.A. M. Tam and C. H. C. Leung, “A multiple media approach to visual information search,” Proc. 1998 ACM SIGIR International Workshop on Multimedia Indexing and Retrieval, Melbourne, August 1998, pp. 1–6.Google Scholar
- 33.A. M. Tam, and C. H. C. Leung, “Semantic content retrieval and structured annotation: beyond keywords”, ISO/IEC JTC1/SC29/WG11 MPEG00/M5738, Noordwijkerhout, NL, March 2000.Google Scholar
- 34.Transaction Processing Performance Council, http://www.tpc.org/.