Interactive Relevance Visual Learning for Image Retrieval

  • Hsin-Chia FuEmail author
  • L. X. Zheng
  • J. B. Wang
  • Hsiao-Tien Pao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9094)


This paper proposes mixture Gaussian neural networks (MGNN) to learn visual features from user specified query image objects or regions for relevance image retrieval. Instead of segmenting query image regions from sample images, relevance feedback feature learning is performed by the proposed MGNN to extract query visual features. After feature learning, the MGNN can be used to measure the appearance difference between the query features and images for image retrieval. The proposed methods were tested on COREL image gallery and the WWW image collections, and testing results were compared with currently leading approaches. From the experimental results, that the extracted and learned query visual features by MGNN can be very close to users’ mind and/or desire, and the closeness is somewhat related to the number of feature leaning iterations. Since any dimensional data can be approximated by mixture Gaussian distributions, thus using MGNN to query and to retrieve similar and/or relevance high dimensional data or images will be a new area of research for future works.


Content-based image retrieval Visual keywords Mixture gaussian distribution Reinforced and anti-reinforced learning Decision-based neural network 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Shen, H.T., Ooi, B.C., Tan, K.-L.: Giving meanings to www images. In: MULTIMEDIA 2000: Proceedings of the Eighth ACM International Conference on Multimedia, New York, NY, USA, pp. 39–47 (2000)Google Scholar
  2. 2.
    Eakins, J., Graham, M.: Content-based image retrieval. In: Technical Report. University of Northumbria at Newcastle (1999)Google Scholar
  3. 3.
    Sethi, I.K., Coman, I.L.: Mining association rules between low-level image features and high-level concepts. In: Proceedings of the SPIE Data Mining and Knowledge Discovery, vol. III, pp. 279–290 (2001)Google Scholar
  4. 4.
    Chang, S.K., Liu, S.H.: Picture indexing and abstraction techniques for pictorial databases. IEEE Trans. Pattern Anal. Mach. Intell. 6(4), 475–483 (1984)zbMATHCrossRefGoogle Scholar
  5. 5.
    Faloutsos, C., Barber, R., Flickner, M., Hafner, J., Niblack, W., Petkovic, D., Equitz, W.: Efficient and effective querying by image content. J. Intell. Inf. Syst. 3(3–4), 231–262 (1994)CrossRefGoogle Scholar
  6. 6.
    Pentland, A., Picard, R.W., Scaroff, S.: Photobook: content-based manipulation for image databases. Int. J. Comput. Vision 18(3), 233–254 (1996)CrossRefGoogle Scholar
  7. 7.
    Gupta, A., Jain, R.: Visual information retrieval. Commun. ACM 40(5), 70–79 (1997)CrossRefGoogle Scholar
  8. 8.
    Smith, J.R., Chang, S.F.: VisualSeek: a fully automatic content based query system. In: Proceedings of the Fourth ACM International Conference on Multimedia, pp. 87–98 (1996)Google Scholar
  9. 9.
    Ma, W.Y., Manjunath, B.: Netra: a toolbox for navigating large image databases. In: Proceedings of the IEEE International Conference on Image Processing, pp. 568–571 (1997)Google Scholar
  10. 10.
    Wang, J.Z., Li, J., Wiederhold, G.: SIMPLIcity: semantics-sensitive integrated matching for picture libraries. IEEE Trans. Pattern Anal. Mach. Intell. 23(9), 947–963 (2001)CrossRefGoogle Scholar
  11. 11.
    Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys 40(2), 5:1–5:60 (2008)CrossRefGoogle Scholar
  12. 12.
    Srihari, R.K., Zhang, Z., Rao, A., Baird, H., Chen, F.: Intelligent indexing and semantic retrieval of multimodal documents. Information Retrieval 2, 245–275 (1999)CrossRefGoogle Scholar
  13. 13.
    Jing, F., Li, M., Zhang, L., Zhang, H.-J., Zhang, B.: Learning in region based image retrieval. In: Bakker, E.M., Lew, M., Huang, T.S., Sebe, N., Zhou, X.S. (eds.) CIVR 2003. LNCS, vol. 2728. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  14. 14.
    Carson, C., Thomas, M., Belongie, S., Hellerstein, J.M., Malik, J.: Blobworld: a system for region-based image indexing and retrieval. In: Huijsmans, D.P., Smeulders, A.W. (eds.) VISUAL 1999. LNCS, vol. 1614, pp. 509–517. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  15. 15.
    Rui, Y., Huang, T.S., Ortega, M., Mehrotra, S.: Relevance feedback: A power tool in interactive content-based image retrieval. IEEE Trans. Circ. Syst. Video Technol. 8(5), 644–655 (1998)CrossRefGoogle Scholar
  16. 16.
    Xu, Y.-Y., Tseng, C.-L., Fu, H.-C.: Texture Recognition by Generalized Probabilistic Decision-Based Neural Networks. Expert Systems with Applications: An International Journal (0957-4174) 38(5) (May 2011) Google Scholar
  17. 17.
    Fu, H.-C., Xu, Y.-Y., Pao, H.-T.: Multimodal search for effective image retrieval. In: Proceedings of 15th International Conference on Systems, Signals and Image Processing (IWSSIP 2008), Bratislava, Slovakia, June 25–28, pp. 233–236 (2008)Google Scholar
  18. 18.
    Xu, Y.Y., Chuang, S.C., Tseng, C.L., Fu, H.-C.: Generalized probabilistic decision-based neural networks for texture classification and retrieval. In: The 2010 International Conference on Modelling, Identification and Control (ICMIC), Okayama, Japan, pp. 500–504 (2010)Google Scholar
  19. 19.
    Xu, Y.-Y., Fu, H.-C.: Visual Keyword Based Image Retrieval. in IJCSI 9(3), No 2 (May 2012)Google Scholar
  20. 20.
    Xu, H., Yaguchi, H.: Visual evaluation at scale of threshold to suprathreshold color difference. Color Research and: Application 30(3), 198–208 (2005)CrossRefGoogle Scholar
  21. 21.
    Daugman, J.G.: Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression. IEEE Transactions on Acoustics, Speech, and Signal Processing 36(7), 1169–1179 (1988)zbMATHCrossRefGoogle Scholar
  22. 22.
    Manjuath, B.S., Ma, W.Y.: Texture features for browsing and retrieval of image data. IEEE Transactions on PAMI 18(8), 837–842 (1996)CrossRefGoogle Scholar
  23. 23.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B 39(1), 1–38 (1977)zbMATHMathSciNetGoogle Scholar
  24. 24.
    Fu, H.C., Xu, Y.Y.: Multi-linguistic handwritten character recognition by Bayesian decision-based neural networks. IEEE Transactions on Signal Processing 46(10), 2781–2789 (1998)CrossRefGoogle Scholar
  25. 25.
    Su, Z., Zhang, H., Li, S.Z., Ma, S.: Relevance feedback in content-based image retrieval: Bayesian framework, feature subspaces, and progressive learning. IEEE Transactions on Image Processing 12(8), 924–937 (2003)CrossRefGoogle Scholar
  26. 26.
    Hsu, C.T., Li, C.Y.: Relevance feedback using generalized Bayesian framework with region-based optimization learning. IEEE Transactions on Image Processing 14(10), 1617–1631 (2005)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Hsin-Chia Fu
    • 1
    Email author
  • L. X. Zheng
    • 1
  • J. B. Wang
    • 1
  • Hsiao-Tien Pao
    • 2
  1. 1.College of EngineeringHuaqiao UniversityQuanzhouChina
  2. 2.Department of Management ScienceNational Chiao-Tung UniversityHsinchuTaiwan, ROC

Personalised recommendations