Abstract
In content-based image retrieval (CBIR) with relevance feedback we would like to retrieve relevant images based on their content features and the feedback given by users. In this paper we view CBIR as an Exploration-Exploitation problem and apply a kernel version of the LinRel algorithm to solve it. By using multiple feature extraction methods and utilising the feedback given by users, we adopt a strategy of multiple kernel learning to find a relevant feature space for the kernel LinRel algorithm. We call this algorithm LinRelMKL. Furthermore, when we have access to eye movement data of users viewing images we can enrich our (multiple) feature spaces by using a tensor kernel SVM. When learning in this enriched space we show that we can significantly improve the search results over the LinRel and LinRelMKL algorithms. Our results suggest that the use of exploration-exploitation with multiple feature spaces is an efficient way of constructing CBIR systems, and that when eye movement features are available, they should be used to help improve CBIR.
Chapter PDF
References
Agrawal, R.: Sample mean based index policies with O(log n) regret for the multi-armed bandit problem. Advances in Applied Probability 27(4), 1054–1078 (1995)
Argyriou, A., Micchelli, C.A., Pontil, M.: Learning convex combinations of continuously parameterized basic kernels. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS (LNAI), vol. 3559, pp. 338–352. Springer, Heidelberg (2005)
Auer, P.: Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research 3, 397–422 (2003)
Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine Learning 47(2-3), 235–256 (2002)
Auer, P., Leung, A., Hussain, Z., Shawe-Taylor, J.: Report on using side information for exploration-exploitation trade-offs. PinView FP7-216529 Project Deliverable Report D4.2.1 (December 2009)
Bach, F.R.: Exploring large feature spaces with hierarchical multiple kernel learning. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, vol. 21, pp. 105–112 (2009)
Bach, F.R., Lanckriet, G.R.G., Jordan, M.I.: Multiple kernel learning, conic duality, and the smo algorithm. In: Proceedings of the Twenty-First International Conference on Machine Learning, vol. 6. ACM, New York (2004)
Chen, Y., Zhou, X.S., Huang, T.: One-class SVM for learning in image retrieval. In: Proceedings of International Conference on Image Processing 2001, vol. 1, pp. 34–37 (2001)
Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys 40, 5:1–5:60 (2008)
Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge (VOC 2007) (2007), Results, http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html
Hardoon, D.R., Pasupa, K.: Image ranking with implicit feedback from eye movements. In: Proceedings of ETRA 2010: ACM Symposium on Eye-Tracking Research & Applications, pp. 291–298. ACM, New York (2010)
Hardoon, D.R., Shawe-Taylor, J.: Decomposing the tensor kernel support vector machine for neuroscience data with structure labels. Machine Learning Journal: Special Issue on Learning From Multiple Sources 79(1-2), 29–46 (2010)
Hussain, Z., Pasupa, K., Saunders, C.J., Shawe-Taylor, J.: Basic metric learning. PinView FP7-216529 Project Deliverable Report D3.1 (December 2008)
Laaksonen, J., Viitaniemi, V.: Evaluation of pointer click relevance feedback in picsom. PinView FP7-216529 Project Deliverable Report D1.2 (August 2008)
Lanckriet, G.R.G., Cristianini, N., Bartlett, P., Ghaoui, L.E., Jordan, M.I.: Learning the kernel matrix with semidefinite programming. Journal of Machine Learning Research 5, 27–72 (2004)
Pasupa, K., Saunders, C., Szedmak, S., Klami, A., Kaski, S., Gunn, S.: Learning to rank images from eye movements. In: HCI 2009: Proceeding of the IEEE 12th International Conference on Computer Vision (ICCV 2009) Workshops on Human-Computer Interaction, pp. 2009–2016 (2009)
Pulmannová, S.: Tensor products of hilbert space effect algebras. Reports on Mathematical Physics 53(2), 301–316 (2004)
Rocchio, J.: Relevance Feedback in Information Retrieval, pp. 313–323 (1971)
Rui, Y., Huang, T.: Optimizing learning in image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 236–243 (2000)
Saunders, C., Klami, A.: Database of eye-movement recordings. Technical Report Project Deliverable Report D8.3, PinView FP7-216529 (2008), http://www.pinview.eu/deliverables.php
Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)
Szedmak, S., Shawe-Taylor, J., Parado-Hernandez, E.: Learning via linear operators: Maximum margin regression; multiclass and multiview learning at one-class complexity. Technical report, University of Southampton (2005)
L. Tobii Technology. Tobii Studio Help, http://studiohelp.tobii.com/StudioHelp_1.2/
Tong, S., Chang, E.: Support vector machine active learning for image retrieval. In: MULTIMEDIA 2001: Proceedings of the Ninth ACM International Conference on Multimedia, pp. 107–118. ACM, New York (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hussain, Z., Leung, A.P., Pasupa, K., Hardoon, D.R., Auer, P., Shawe-Taylor, J. (2010). Exploration-Exploitation of Eye Movement Enriched Multiple Feature Spaces for Content-Based Image Retrieval. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2010. Lecture Notes in Computer Science(), vol 6321. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15880-3_41
Download citation
DOI: https://doi.org/10.1007/978-3-642-15880-3_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15879-7
Online ISBN: 978-3-642-15880-3
eBook Packages: Computer ScienceComputer Science (R0)