Unifying Content and Context Similarities of the Textual and Visual Information in an Image Clustering Framework

  • Bashar Tahayna
  • Saadat M. Alashmi
  • Mohammed Belkhatir
  • Khaled Abbas
  • Yandan Wang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6297)


Content-based image retrieval (CBIR) has been a challenging problem and its performance relies on the efficiency in modeling the underlying content and the similarity measure between the query and the retrieved images. Most existing metrics evaluate pairwise image similarity based only on image content, which is denoted as content similarity. However, other schemes utilize the annotations and the surrounding text to improve the retrieval results. In this study we refer to content as the visual and the textual information belonging to an image. We propose a representation of an image surrounding text in terms of concepts by utilizing an online knowledge source, e.g., Wikipedia, and propose a similarity metric that takes into account the new conceptual representation of the text. Moreover, we combine the content information with the contexts of an image to improve the retrieval percentage. The context of an image is built by constructing a vector with each dimension representing the content (visual and textual/conceptual) similarity between the image and any image in the collection. The context similarity between two images is obtained by computing the similarity between the corresponding context vectors using the vector similarity functions. Then, we fuse the similarity measures into a unified measure to evaluate the overall image similarity. Experimental results demonstrate that the new text representation and the use of the context similarity can significantly improve the retrieval performance.


Clustering Classification Content-based Similarity measures bipitrate graphs 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. ACM Press, a Division of the Association for Computing Machinery (1999)Google Scholar
  2. 2.
    Cai, D., He, X., Li, Z., Wen, J.: Hierarchical Clustering of www Image Search Results Using Visual, Textual and Link Information. In: ACM Multimedia 2004 (2004)Google Scholar
  3. 3.
    Halkidi, M., Batistakis, Y., Vazirgiannis, M.: On Clustering Validation Techniques. Journal of Intelligent Information Systems 17(2-3), 107–145 (2001)zbMATHCrossRefGoogle Scholar
  4. 4.
    Mao, J., Jain, A.K.: A Self-organizing Network for Hyperellipsoidal Clustering (hec). IEEE Transactions on Neural Networks 7(1), 16–29 (1996)CrossRefGoogle Scholar
  5. 5.
    Stumme, G., Taouil, R., Bastide, Y., Pasquier, N., Lakhal, L.: Computing Iceberg Concept Lattices with Titanic. Data & Knowledge Engineering 42(2), 189–222 (2002)zbMATHCrossRefGoogle Scholar
  6. 6.
    Yanai, K.: Generic Image Classification Using Visual Knowledge on the Web. In: Proceedings of the 11th ACM MM, pp. 167–176 (2003)Google Scholar
  7. 7.
    Zhang, D.S., Lu, G.: Generic Fourier Descriptors for Shape-based Image Retrieval. In: Proceedings of IEEE Int. Conf. on Multimedia and Expo., vol. 1, pp. 425–428 (2002)Google Scholar
  8. 8.
    Cui, J., Wen, F., Tang, X.: Real time google and live image search re-ranking. In: Proceeding of the 16th ACM International Conference on Multimedia, pp. 729–732 (2008)Google Scholar
  9. 9.
    Fergus, R., Perona, P., Zisserman, A.: A Visual Category Filter for Google Images. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 242–256. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  10. 10.
    Tahayna, B., Belkhatir, M., Wang, Y.: Clustering of Retrieved Images by Integrating Perceptual Signal Features within Keyword-Based Image Search Engines. In: Proceedings of the 10th Pacific Rim Conference on Multimedia, PCM 2010 (2009)Google Scholar
  11. 11.
    Gao, Y., Fan, J., Luo, H., Satoh, S.: A Novel Approach for Filtering Junk Images from Google Search Results. In: Satoh, S., Nack, F., Etoh, M. (eds.) MMM 2008. LNCS, vol. 4903, pp. 1–12. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  12. 12.
    Cai, D., He, X., Li, Z., Ma, W., Wen, J.: Hierarchical Clustering of WWW Image Search Results Using Visual, Textual and Link Information. In: ACM Multimedia 2004 (2004)Google Scholar
  13. 13.
    Zhao, R., Grosky, W.I.: Narrowing the Semantic Gap- Improved Text-Based Web Document Retrieval Using Visual Features. IEEE Transactions on Multimedia 4(2) (2002)Google Scholar
  14. 14.
    Gao, B., Liu, T.-Y., Qin, T., Zheng, X., Cheng, Q.-S., Ma, Y.-M.: Web Image Clustering by Consistent Utilization of Visual Features and Surrounding Texts. In: Proceeding of the 16th ACM International Conference on Multimedia (2005)Google Scholar
  15. 15.
    Li, Z., Xu, G., Li, M., Ma, W., Zhang, H.: Group WWW image search results by novel inhomogeneous clustering method. In: Proceedings of MMM 2004 (2004)Google Scholar
  16. 16.
    Qiu, G.: Image and Feature Co-clustering. In: Intl. conf on pattern recognition, ICPR, (4), pp. 991–994 (2004)Google Scholar
  17. 17.
    Gao, B., Liu, T., Zheng, X., Cheng, Q., Ma, W.: Consistent Bipartite Graph Co- Partitioning for Star-Structured High-Order Heterogeneous Data Co-Clustering. In: Proceedings of ACM SIGKDD (2005)Google Scholar
  18. 18.
    Ayyasamy, R.-K., Tahayna, B., Alhashmi, S., Eu-gene, S., Egerton, S.: Mining Wikipedia knowledge to improve Document indexing and classification. In: Int. conference on Information Systems, Signal processing and its applications, ISSPA 2010 (2010)Google Scholar
  19. 19.
    Ding, C., He, X., Zha, H., Gu, M., Simon, H.: A min-max cut algorithm for graph partitioning and data clustering. In: Proc. IEEE Int’ l. Conf. Data Mining (2001)Google Scholar
  20. 20.
    Hagen, L., Kahng, A.B.: New spectral methods for ratio cut partitioning and clustering. IEEE. Trans. on Computed Aided Design 11, 1074–1085 (1992)CrossRefGoogle Scholar
  21. 21.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 888–905 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Bashar Tahayna
    • 1
  • Saadat M. Alashmi
    • 1
  • Mohammed Belkhatir
    • 2
  • Khaled Abbas
    • 3
  • Yandan Wang
    • 1
  1. 1.Monash University
  2. 2.Université Claude Bernard Lyon 1France
  3. 3.University of MalayaMalaysia

Personalised recommendations