Skip to main content
SpringerLink
Log in

Mental image search by boolean composition of region categories

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Existing content-based image retrieval paradigms almost never address the problem of starting the search, when the user has no starting example image but rather a mental image. We propose a new image retrieval system to allow the user to perform mental image search by formulating boolean composition of region categories. The query interface is a region photometric thesaurus which can be viewed as a visual summary of salient regions available in the database. It is generated from the unsupervised clustering of regions with similar visual content into categories. In this thesaurus, the user simply selects the types of regions which should and should not be present in the mental image (boolean composition). The natural use of inverted tables on the region category labels enables powerful boolean search and very fast retrieval in large image databases. The process of query and search of images relates to that of documents with Google. The indexing scheme is fully unsupervised and the query mode requires minimal user interaction (no example image to provide, no sketch to draw). We demonstrate the feasibility of such a framework to reach the user mental target image with two applications: a photo-agency scenario on Corel Photostock and a TV news scenario. Perspectives will be proposed for this simple and innovative framework, which should motivate further development in various research areas.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Notes

  1. Google: http://www.google.com

  2. The idea of the “mental document” may be more or less precise in the user's mind: it may be an already seen document or more generally a document related to a particular topic.

References

  1. Baeza-Yates R, Ribeiro-Neto B (1999) Modern information retrieval. Addison-Wesley

  2. Bezdek JC (1981) Pattern recognition with fuzzy objective functions. Plenum, New York NY

    Google Scholar 

  3. Boujemaa N, Fauqueur J, Ferecatu M, Fleuret F, Gouet V, Le Saux B, Sahbi H (2001) Ikona: Interactive generic and specific image retrieval. International workshop on Multimedia Content-Based Indexing and Retrieval (MMCBIR), Rocquencourt, France, pp 25–28

  4. Carson C, et al (1999) Blobworld: A system for region-based image indexing and retrieval. Proc. of International Conference on Visual Information System, LNCS 1614:509–517

    Google Scholar 

  5. Cox IJ, Miller ML, Minka TP (2000) The bayesian image retrieval system, pichunter: Theory, implementation and psychological experiments. IEEE Trans Image Process 9(1):20–37

    Article  Google Scholar 

  6. DelBimbo A, Pala P (1997, February) Visual image retrieval by elastic matching of user sketches. IEEE Trans Pattern Anal Mach Intell 19(2):121–132

    Article  Google Scholar 

  7. DelBimbo A, Vicario E (1998, June) Using weighted spatial relationships in retrieval by visual contents. IEEE workshop on Image and Video Libraries

  8. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. JR Stat Soc 39:1–38

    MATH  MathSciNet  Google Scholar 

  9. Deng Y, Manjunath BS (1999, March) An efficient low-dimensional color indexing scheme for region based image retrieval. Proc. IEEE Int Conf Acoust Speech Signal Proc (ICASSP), Phoenix, Arizona

  10. Egenhofer MJ (1997) Query processing in spatial query by sketch. J Vis Lang Comput (JVLC) 8(4):403–424

    Article  Google Scholar 

  11. Fauqueur J (2003) Contributions to image retrieval by their visual components. PhD Thesis, UVSQ - INRIA, (in French).

  12. Fauqueur J, Boujemaa N, (2003) Logical query composition from local visual feature thesaurus. International Workshop on Content-Based Multimedia Indexing (CBMI), Rennes, France

  13. Fauqueur J, Boujemaa N (2004) Region-based image retrieval: Fast coarse segmentation and fine color description. J Vis Lang Comput (JVLC), special issue on Visual Information Systems 15(1):69–95

    Article  Google Scholar 

  14. Flickner M, et al (1995) Query by image and video content: The qbic system. IEEE Computer 28(9):23–32

    Google Scholar 

  15. Frigui H, Krishnapuram R (1997) Clustering by competitive agglomeration. Pattern Recognition 30(7): 1109–1119

    Article  Google Scholar 

  16. Fung CY, Loe KJ (1999) Learning primitive and scene semantics for image for classification and retrieval. ACM Multimedia

  17. Gouet V, Boujemaa N (2001) Object-based queries using color points of interest. IEEE Workshop on Content-Based Access of Image and Video Libraries (CBAIVL)

  18. Gupta A, et al (1996) The virage image search engine: an open framework for image management. SPIE Storage and Retrieval for Image and Video Databases, 2670

  19. Harper DJ, Jose JM, Furner J (1998) Spatial querying for image retrieval: A user-oriented evaluation. International ACM SIGIR conference, pp 232–240

  20. Hiroike A, Musha Y, Sugimoto A, Mori Y (1999) Visualization of information spaces to retrieve and browse image data. International Conference on Visual Information System (VIS)

  21. Huang T, Mehrotra S, Ramchandran K (1996) Multimedia analysis and retrieval system (mars) project. Proc. of the 33rd Annual Clinic on Library Application of Data Processing—Digital Image Access and Retrieval

  22. Huang T, Rui Y, Mehrotra S (1997) Content-based image retrieval with relevance feedback in mars. IEEE Int Conf Image Proc (ICIP)

  23. Kohonen T (1997) Self-organizing maps. Springer Berlin Heidelberg New York

    Google Scholar 

  24. La Cascia M, Sethi S, Sclaroff S (1998, June ) Combining textual and visual cues for content-based image retrieval on the world wide web. IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL)

  25. Laaksonen J, Oja E, Koskela M, Brandt S (2000) Analyzing low-level visual features using content-based image retrieval. International Conference on Neural Information Processing (ICONIP). Taejon, Korea

  26. LeSaux B, Boujemaa N (2002) Unsupervised robust clustering for image database categorization. IAPR Int Conf Pattern Recognit (ICPR)

  27. Lim JH (1999) Learnable visual keywords for image classification. ACM conference on Digital libraries, pp 139–145

  28. Linde Y, Buzo A, Gray RM (1980) An algorithm for vector quantizer design. IEEE Trans Commun COM-28:84–95

    Article  Google Scholar 

  29. Ma WY, Manjunath BS (1998) A texture thesaurus for browsing large aerial photographs. Journal of the American Society of Information Science 49(7):633–648

    Article  Google Scholar 

  30. Ma WY, Manjunath BS (1999) Netra: A toolbox for navigating large image databases. Multimedia Syst 7(3):184–198

    Article  Google Scholar 

  31. MacDonald S, Tait (2003) Search strategies in content-based image retrieval. International ACM SIGIR conference

  32. MacQueen J (1967) Some methods for classification and analysis of multivariate observations. Proc. of the Fifth Berkeley Symp on Math Stat and Prob 1:281–296

    MATH  MathSciNet  Google Scholar 

  33. Malki J, Boujemaa N, Nastar C, Winter A (1999) Region queries without segmentation for image retrieval by content. In Proc. of International Conference on Visual Information System (VIS), pp 115–122

  34. Manjunath BS, Salembier P, Sikora T (2002) Introduction to MPEG-7: Multimedia Content Description Interface. Wiley, ISBN: 0-471-48678-7

  35. Meiers T, Sikora T, Keller I (2002) Hierarchical image database browsing environment with embedded relevance feedback. IEEE Int Conf Image Proc (ICIP)

  36. Meilhac C, Nastar C (1999) Relevance feedback and category search in image databases. IEEE International Conference on Multimedia Computing and Systems

  37. Moghaddam B, Biermann H, Margaritis D (1999) Defining image content with multiple regions of interest. IEEE Workshop on Content-Based Access of Image and Video Libraries (CBAIVL)

  38. Nastar C, Mitschke M, Meilhac C, Boujemaa N (1998) Surfimage: A flexible content-based image retrieval system. ACM Multimedia Conference Proceedings, Bristol, UK

  39. Niblack W, Barber R, Equitz W, Flickner M, et al (1993) The QBic project: Querying images by content using color, texture, and shape. Proc. SPIE (Storage and Retrieval for Image and Video Databases) 1908:173–187

    Google Scholar 

  40. Pentland A, Picard R, Sclaroff S (1994, February) Photobook: content-based manipulation of image databases. SPIE Storage and Retrieval for Image and Video Databases, II(2185)

  41. Picard RW (1995) Toward a visual thesaurus. MIT Technical Report TR358

  42. Rissanen J (1978) Modeling by shortest data description. Automatica

  43. Rodden K, Basalaj W, Sinclair D, Wood K (2001) Does organisation by similarity assist image browsing? International ACM SIGCHI conference, pp 190–197

  44. Rubner Y (1999) Perceptual metrics for image database navigation. PhD Thesis, Stanford University

  45. Sclaroff S, Taycher L, La Cascia M (1997, June) Imagerover: A content-based image browser for the world wide web. IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL)

  46. Sivic J, Zisserman A (2003) Video google: A text retrieval approach to object matching in videos. Proceedings International Conference on Computer Vision (ICCV), pp 1470–1477

  47. Smeulders A, Worring M, Santini S, Gupta A, Jain R (2000) Content based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell (PAMI) 22(12):1349–1380

    Article  Google Scholar 

  48. Smith JR, Chang SF (1996) Tools and techniques for color image retrieval. IST/SPIE Proceedings, pp 426–437

  49. Smith JR, Chang SF (1996) Visualseek: A fully automated content-based image query system. ACM Multimedia Conference Boston, MA, USA, pp 87–98

  50. Squire D, Muller W, Muller H, Raki J (1999) Content-based query of image databases, inspirations from text retrieval: inverted files, frequency-based weights and relevance feedback. 11th Scandinavian Conference on Image Analysis (SCIA) Kangerlussuaq, Greenland

  51. Swain M, Ballard D (1991) Color indexing. Int J Comput Vis (IJCV) 7(1):11–32

    Article  Google Scholar 

  52. Town C, Sinclair D (2001) Content based image retrieval using semantic visual categories. ATT Technical Report

  53. Wang JZ, Du Y (2001) Rf*ipf: A weighting scheme for multimedia information retrieval. IEEE International Conference on Image Analysis and Processing (ICIAP)

  54. Witten IH, Moffat A, Bell TC (1994) Managing gigabytes: compressing and indexing documents and images. Van Nostrand Reinhold, 115 Fifth Avenue, New York, NY 10003, USA

    Google Scholar 

  55. Zhang HJ, Jing F, Li M, Zhang B (2002) An effective region-based image retrieval framework. Proceeding of ACM Multimedia, pp 456–465

Download references

Acknowledgments

We would like to thank Aicha El Golli for generating the Kohonen maps of the visual thesaurus. We would also like to thank TF1 Channel for providing TV news videos.

Author information

Authors and Affiliations

  1. Projet IMEDIA - INRIA, BP 105, 78153, Le Chesnay, Cedex, France

    Julien Fauqueur & Nozha Boujemaa

Authors
  1. Julien Fauqueur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nozha Boujemaa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Julien Fauqueur.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fauqueur, J., Boujemaa, N. Mental image search by boolean composition of region categories. Multimed Tools Appl 31, 95–117 (2006). https://doi.org/10.1007/s11042-006-0033-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-006-0033-3

Keywords

Search

Navigation