Abstract
We propose Information-Theoretic Active Learning (ITAL), a novel batch-mode active learning method for binary classification, and apply it for acquiring meaningful user feedback in the context of content-based image retrieval. Instead of combining different heuristics such as uncertainty, diversity, or density, our method is based on maximizing the mutual information between the predicted relevance of the images and the expected user feedback regarding the selected batch. We propose suitable approximations to this computationally demanding problem and also integrate an explicit model of user behavior that accounts for possible incorrect labels and unnameable instances. Furthermore, our approach does not only take the structure of the data but also the expected model output change caused by the user feedback into account. In contrast to other methods, ITAL turns out to be highly flexible and provides state-of-the-art performance across various datasets, such as MIRFLICKR and ImageNet.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ayache, S., Quénot, G.: Evaluation of active learning strategies for video indexing. Sig. Process.: Image Commun. 22(7), 692–704 (2007)
Brinker, K.: Incorporating diversity in active learning with support vector machines. In: International Conference on Machine Learning (ICML), pp. 59–66 (2003)
Cardoso, T.N., Silva, R.M., Canuto, S., Moro, M.M., Gonçalves, M.A.: Ranked batch-mode active learning. Inf. Sci. 379, 313–337 (2017)
Cox, I.J., Miller, M.L., Minka, T.P., Papathomas, T.V., Yianilos, P.N.: The Bayesian image retrieval system, pichunter: theory, implementation, and psychophysical experiments. IEEE Trans. Image Process. 9(1), 20–37 (2000)
Demir, B., Bruzzone, L.: A novel active learning method in relevance feedback for content-based remote sensing image retrieval. IEEE Trans. Geosci. Remote Sens. 53(5), 2323–2334 (2015)
Deselaers, T., Paredes, R., Vidal, E., Ney, H.: Learning weighted distances for relevance feedback in image retrieval. In: International Conference on Pattern Recognition (ICPR), pp. 1–4. IEEE (2008)
Fei-Fei, L., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 524–531. IEEE (2005)
Freytag, A., Rodner, E., Bodesheim, P., Denzler, J.: Labeling examples that matter: relevance-based active learning with Gaussian processes. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 282–291. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40602-7_31
Freytag, A., Rodner, E., Denzler, J.: Selecting influential examples: active learning with expected model output changes. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 562–577. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_37
Friedman, J., Hastie, T., Tibshirani, R.: Example: ZIP code data (Ch. 11.7). In: The Elements of Statistical Learning. Springer Series in Statistics, New York (2001)
Genz, A.: Numerical computation of multivariate normal probabilities. J. Comput. Graph. Stat. 1(2), 141–149 (1992)
Giang, N.T., Tao, N.Q., Dung, N.D., The, N.T.: Batch mode active learning for interactive image retrieval. In: 2014 IEEE International Symposium on Multimedia (ISM), pp. 28–31. IEEE (2014)
Guestrin, C., Krause, A., Singh, A.P.: Near-optimal sensor placements in Gaussian processes. In: International Conference on Machine Learning (ICML), pp. 265–272. ACM (2005)
Guo, Y., Greiner, R.: Optimistic active-learning using mutual information. In: IJCAI, vol. 7, pp. 823–829 (2007)
Guo, Y., Schuurmans, D.: Discriminative batch mode active learning. In: Advances in Neural Information Processing Systems (NIPS), pp. 593–600 (2008)
Huiskes, M.J., Lew, M.S.: The MIR flickr retrieval evaluation. In: Proceedings of the 2008 ACM International Conference on Multimedia Information Retrieval, MIR 2008. ACM, New York (2008)
Jain, P., Kapoor, A.: Active learning for large multi-class problems. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 762–769. IEEE (2009)
Johns, E., Mac Aodha, O., Brostow, G.J.: Becoming the expert - interactive multi-class machine teaching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2616–2624. IEEE (2015)
Käding, C., Freytag, A., Rodner, E., Bodesheim, P., Denzler, J.: Active learning and discovery of object categories in the presence of unnameable instances. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4343–4352 (2015)
Käding, C., Rodner, E., Freytag, A., Mothes, O., Barz, B., Denzler, J.: Active learning for regression tasks with expected model output changes. In: British Machine Vision Conference (BMVC) (2018)
Kapoor, A., Grauman, K., Urtasun, R., Darrell, T.: Active learning with Gaussian processes for object categorization. In: IEEE International Conference on Computer Vision (ICCV), pp. 1–8. IEEE (2007)
Krause, A., Guestrin, C.: Nonmyopic active learning of Gaussian processes: an exploration-exploitation approach. In: International Conference on Machine Learning (ICML), pp. 449–456. ACM (2007)
Li, X., Guo, Y.: Adaptive active learning for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 859–866 (2013)
Lütz, A., Rodner, E., Denzler, J.: I want to know more–efficient multi-class incremental learning using Gaussian processes. Pattern Recogn. Image Anal. 23(3), 402–407 (2013)
Niblack, C.W., et al.: QBIC project: querying images by content, using color, texture, and shape. In: Storage and Retrieval for Image and Video Databases, vol. 1908, pp. 173–188. International Society for Optics and Photonics (1993)
Rasmussen, C.E., Williams, C.K.: Gaussian Processes for Machine Learning, vol. 1. MIT Press, Cambridge (2006)
Rodner, E., Freytag, A., Bodesheim, P., Fröhlich, B., Denzler, J.: Large-scale Gaussian process inference with generalized histogram intersection kernels for visual recognition tasks. Int. J. Comput. Vis. 121(2), 253–280 (2017)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Smeulders, A.W., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 22(12), 1349–1380 (2000)
Tong, S., Chang, E.: Support vector machine active learning for image retrieval. In: ACM International Conference on Multimedia, pp. 107–118. ACM (2001)
Yang, Y., Ma, Z., Nie, F., Chang, X., Hauptmann, A.G.: Multi-class active learning by uncertainty sampling with diversity maximization. Int. J. Comput. Vis. 113(2), 113–127 (2015)
Zhu, J., Wang, H., Yao, T., Tsou, B.K.: Active learning with sampling by uncertainty and density for word sense disambiguation and text classification. In: International Conference on Computational Linguistics, vol. 1 (2008)
Acknowledgements
This work was supported by the German Research Foundation as part of the priority programme “Volunteered Geographic Information: Interpretation, Visualisation and Social Computing” (SPP 1894, contract number DE 735/11-1).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Barz, B., Käding, C., Denzler, J. (2019). Information-Theoretic Active Learning for Content-Based Image Retrieval. In: Brox, T., Bruhn, A., Fritz, M. (eds) Pattern Recognition. GCPR 2018. Lecture Notes in Computer Science(), vol 11269. Springer, Cham. https://doi.org/10.1007/978-3-030-12939-2_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-12939-2_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-12938-5
Online ISBN: 978-3-030-12939-2
eBook Packages: Computer ScienceComputer Science (R0)