Semantic Image Clustering Using Object Relation Network

  • Na Chen
  • Viktor K. Prasanna
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7633)


This paper presents a novel method to organize a collection of images into a hierarchy of clusters based on image semantics. Given a group of raw images with no metadata as input, our method describes the semantics of each image with a bag-of-semantics model (i.e., a set of meaningful descriptors), which is derived from the image’s Object Relation Network [5] - an expressive graph model representing rich semantics for image objects and their relations. We adopt the class hierarchies in a guide ontology as different levels of lenses to view the bag-of-semantics models. Image clusters are automatically extracted by grouping images with the same bag-of-semantics viewed through a certain lens. With a series of coarse-to-fine lenses, images are clustered in a top-down hierarchical manner. In addition, given that users can have different perspectives regarding how images should be clustered, our method allows each user to control the clustering process while browsing, and thus dynamically adjusts the clustering result according to the user’s preferences.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bi, J., Chen, Y., Wang, J.Z.: A sparse support vector machine approach to region-based image categorization. In: CVPR (2005) 4Google Scholar
  2. 2.
    Bosch, A., Zisserman, A., Muñoz, X.: Scene Classification Via pLSA. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part IV. LNCS, vol. 3954, pp. 517–530. Springer, Heidelberg (2006) 1, 4CrossRefGoogle Scholar
  3. 3.
    Cai, D., He, X., Li, Z., Ma, W.Y., Wen, J.R.: Hierarchical clustering of www image search results using visual, textual and link information. ACM Multimedia (2004) 1, 3Google Scholar
  4. 4.
    Chen, N., Prasanna, V.K.: A bag-of-semantics model for image clustering. Tech. rep., University of Southern California (August 2012), 7
  5. 5.
    Chen, N., Zhou, Q.Y., Prasanna, V.: Understanding web images by object relation network. In: Proceedings of the 21st International Conference on World Wide Web (2012) 1, 2, 4Google Scholar
  6. 6.
    Chen, Y., Wang, J.Z.: Image categorization by learning and reasoning with regions. J. Mach. Learn. Res. (2004) 4Google Scholar
  7. 7.
    Chen, Y., Wang, J.Z., Krovetz, R.: Clue: Cluster-based retrieval of images by unsupervised learning. IEEE Transactions on Image Processing (2003) 1, 3Google Scholar
  8. 8.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: CVPR (2009) 6Google Scholar
  9. 9.
    Everingham, M., Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2011 (VOC 2011) Results (2011), 6
  10. 10.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. IEEE TPAMI 32(9) (2010) 6Google Scholar
  11. 11.
    Gao, B., Liu, T.Y., Qin, T., Zheng, X., Cheng, Q.S., Ma, W.Y.: Web image clustering by consistent utilization of visual features and surrounding texts. ACM Multimedia (2005) 1, 3Google Scholar
  12. 12.
    van Gemert, J.C., Geusebroek, J.-M., Veenman, C.J., Smeulders, A.W.M.: Kernel Codebooks for Scene Categorization. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 696–709. Springer, Heidelberg (2008) 1, 4CrossRefGoogle Scholar
  13. 13.
    Gordon, S., Greenspan, H., Goldberger, J.: Applying the information bottleneck principle to unsupervised clustering of discrete and continuous image representations. In: ICCV (2003) 1, 3Google Scholar
  14. 14.
    Jing, F., Wang, C., Yao, Y., Deng, K., Zhang, L., Ma, W.Y.: Igroup: web image search results clustering. ACM Multimedia (2006) 3Google Scholar
  15. 15.
    Li, F.F., Perona, P.: A bayesian hierarchical model for learning natural scene categories. In: CVPR (2005) 1, 4Google Scholar
  16. 16.
    Liu, Y., Chen, X., Zhang, C., Sprague, A.: Semantic clustering for region-based image retrieval. J. Vis. Comun. Image Represent. (2009) 4Google Scholar
  17. 17.
    Rodden, K., Basalaj, W., Sinclair, D., Wood, K.: Does organisation by similarity assist image browsing? In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2001) 1, 3Google Scholar
  18. 18.
    Wang, X.J., Ma, W.Y., Zhang, L., Li, X.: Iteratively clustering web images based on link and attribute reinforcements. ACM Multimedia (2005) 1, 3Google Scholar
  19. 19.
    Zheng, X., Cai, D., He, X., Ma, W.Y., Lin, X.: Locality preserving clustering for image database. ACM Multimedia (2004) 3Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Na Chen
    • 1
  • Viktor K. Prasanna
    • 1
  1. 1.University of Southern CaliforniaUSA

Personalised recommendations