Information-Theoretic Active Learning for Content-Based Image Retrieval

  • Björn BarzEmail author
  • Christoph Käding
  • Joachim Denzler
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11269)


We propose Information-Theoretic Active Learning (ITAL), a novel batch-mode active learning method for binary classification, and apply it for acquiring meaningful user feedback in the context of content-based image retrieval. Instead of combining different heuristics such as uncertainty, diversity, or density, our method is based on maximizing the mutual information between the predicted relevance of the images and the expected user feedback regarding the selected batch. We propose suitable approximations to this computationally demanding problem and also integrate an explicit model of user behavior that accounts for possible incorrect labels and unnameable instances. Furthermore, our approach does not only take the structure of the data but also the expected model output change caused by the user feedback into account. In contrast to other methods, ITAL turns out to be highly flexible and provides state-of-the-art performance across various datasets, such as MIRFLICKR and ImageNet.


Batch-mode active learning Image retrieval 



This work was supported by the German Research Foundation as part of the priority programme “Volunteered Geographic Information: Interpretation, Visualisation and Social Computing” (SPP 1894, contract number DE 735/11-1).

Supplementary material

480455_1_En_45_MOESM1_ESM.pdf (2.3 mb)
Supplementary material 1 (pdf 2338 KB)


  1. 1.
    Ayache, S., Quénot, G.: Evaluation of active learning strategies for video indexing. Sig. Process.: Image Commun. 22(7), 692–704 (2007)Google Scholar
  2. 2.
    Brinker, K.: Incorporating diversity in active learning with support vector machines. In: International Conference on Machine Learning (ICML), pp. 59–66 (2003)Google Scholar
  3. 3.
    Cardoso, T.N., Silva, R.M., Canuto, S., Moro, M.M., Gonçalves, M.A.: Ranked batch-mode active learning. Inf. Sci. 379, 313–337 (2017)CrossRefGoogle Scholar
  4. 4.
    Cox, I.J., Miller, M.L., Minka, T.P., Papathomas, T.V., Yianilos, P.N.: The Bayesian image retrieval system, pichunter: theory, implementation, and psychophysical experiments. IEEE Trans. Image Process. 9(1), 20–37 (2000)CrossRefGoogle Scholar
  5. 5.
    Demir, B., Bruzzone, L.: A novel active learning method in relevance feedback for content-based remote sensing image retrieval. IEEE Trans. Geosci. Remote Sens. 53(5), 2323–2334 (2015)CrossRefGoogle Scholar
  6. 6.
    Deselaers, T., Paredes, R., Vidal, E., Ney, H.: Learning weighted distances for relevance feedback in image retrieval. In: International Conference on Pattern Recognition (ICPR), pp. 1–4. IEEE (2008)Google Scholar
  7. 7.
    Fei-Fei, L., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 524–531. IEEE (2005)Google Scholar
  8. 8.
    Freytag, A., Rodner, E., Bodesheim, P., Denzler, J.: Labeling examples that matter: relevance-based active learning with Gaussian processes. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 282–291. Springer, Heidelberg (2013). Scholar
  9. 9.
    Freytag, A., Rodner, E., Denzler, J.: Selecting influential examples: active learning with expected model output changes. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 562–577. Springer, Cham (2014). Scholar
  10. 10.
    Friedman, J., Hastie, T., Tibshirani, R.: Example: ZIP code data (Ch. 11.7). In: The Elements of Statistical Learning. Springer Series in Statistics, New York (2001)Google Scholar
  11. 11.
    Genz, A.: Numerical computation of multivariate normal probabilities. J. Comput. Graph. Stat. 1(2), 141–149 (1992)Google Scholar
  12. 12.
    Giang, N.T., Tao, N.Q., Dung, N.D., The, N.T.: Batch mode active learning for interactive image retrieval. In: 2014 IEEE International Symposium on Multimedia (ISM), pp. 28–31. IEEE (2014)Google Scholar
  13. 13.
    Guestrin, C., Krause, A., Singh, A.P.: Near-optimal sensor placements in Gaussian processes. In: International Conference on Machine Learning (ICML), pp. 265–272. ACM (2005)Google Scholar
  14. 14.
    Guo, Y., Greiner, R.: Optimistic active-learning using mutual information. In: IJCAI, vol. 7, pp. 823–829 (2007)Google Scholar
  15. 15.
    Guo, Y., Schuurmans, D.: Discriminative batch mode active learning. In: Advances in Neural Information Processing Systems (NIPS), pp. 593–600 (2008)Google Scholar
  16. 16.
    Huiskes, M.J., Lew, M.S.: The MIR flickr retrieval evaluation. In: Proceedings of the 2008 ACM International Conference on Multimedia Information Retrieval, MIR 2008. ACM, New York (2008)Google Scholar
  17. 17.
    Jain, P., Kapoor, A.: Active learning for large multi-class problems. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 762–769. IEEE (2009)Google Scholar
  18. 18.
    Johns, E., Mac Aodha, O., Brostow, G.J.: Becoming the expert - interactive multi-class machine teaching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2616–2624. IEEE (2015)Google Scholar
  19. 19.
    Käding, C., Freytag, A., Rodner, E., Bodesheim, P., Denzler, J.: Active learning and discovery of object categories in the presence of unnameable instances. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4343–4352 (2015)Google Scholar
  20. 20.
    Käding, C., Rodner, E., Freytag, A., Mothes, O., Barz, B., Denzler, J.: Active learning for regression tasks with expected model output changes. In: British Machine Vision Conference (BMVC) (2018)Google Scholar
  21. 21.
    Kapoor, A., Grauman, K., Urtasun, R., Darrell, T.: Active learning with Gaussian processes for object categorization. In: IEEE International Conference on Computer Vision (ICCV), pp. 1–8. IEEE (2007)Google Scholar
  22. 22.
    Krause, A., Guestrin, C.: Nonmyopic active learning of Gaussian processes: an exploration-exploitation approach. In: International Conference on Machine Learning (ICML), pp. 449–456. ACM (2007)Google Scholar
  23. 23.
    Li, X., Guo, Y.: Adaptive active learning for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 859–866 (2013)Google Scholar
  24. 24.
    Lütz, A., Rodner, E., Denzler, J.: I want to know more–efficient multi-class incremental learning using Gaussian processes. Pattern Recogn. Image Anal. 23(3), 402–407 (2013)CrossRefGoogle Scholar
  25. 25.
    Niblack, C.W., et al.: QBIC project: querying images by content, using color, texture, and shape. In: Storage and Retrieval for Image and Video Databases, vol. 1908, pp. 173–188. International Society for Optics and Photonics (1993)Google Scholar
  26. 26.
    Rasmussen, C.E., Williams, C.K.: Gaussian Processes for Machine Learning, vol. 1. MIT Press, Cambridge (2006)Google Scholar
  27. 27.
    Rodner, E., Freytag, A., Bodesheim, P., Fröhlich, B., Denzler, J.: Large-scale Gaussian process inference with generalized histogram intersection kernels for visual recognition tasks. Int. J. Comput. Vis. 121(2), 253–280 (2017)CrossRefGoogle Scholar
  28. 28.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  30. 30.
    Smeulders, A.W., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 22(12), 1349–1380 (2000)CrossRefGoogle Scholar
  31. 31.
    Tong, S., Chang, E.: Support vector machine active learning for image retrieval. In: ACM International Conference on Multimedia, pp. 107–118. ACM (2001)Google Scholar
  32. 32.
    Yang, Y., Ma, Z., Nie, F., Chang, X., Hauptmann, A.G.: Multi-class active learning by uncertainty sampling with diversity maximization. Int. J. Comput. Vis. 113(2), 113–127 (2015)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Zhu, J., Wang, H., Yao, T., Tsou, B.K.: Active learning with sampling by uncertainty and density for word sense disambiguation and text classification. In: International Conference on Computational Linguistics, vol. 1 (2008)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Friedrich Schiller University JenaJenaGermany

Personalised recommendations