Interactive Event Search through Transfer Learning

  • Antony Lam
  • Amit K. Roy-Chowdhury
  • Christian R. Shelton
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6494)


Activity videos are widespread on the Internet but current video search is limited to text tags due to limitations in recognition systems. One of the main reasons for this limitation is the wide variety of activities users could query. Thus codifying knowledge for all queries becomes problematic. Relevance Feedback (RF) is a retrieval framework that addresses this issue via interactive feedback with the user during the search session. An added benefit is that RF can also learn the subjective component of a user’s search preferences. However for good retrieval performance, RF may require a large amount of user feedback for activity search. We address this issue by introducing Transfer Learning (TL) into RF. With TL, we can use auxiliary data from known classification problems different from the user’s target query to decrease the needed amount of user feedback. We address key issues in integrating RF and TL and demonstrate improved performance on the challenging YouTube Action Dataset.


Image Retrieval Average Precision Relevance Feedback User Feedback Target Task 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Burges, C.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2, 121–167 (1998)CrossRefGoogle Scholar
  2. 2.
    Canu, S., Grandvalet, Y., Guigue, V., Rakotomamonjy, A.: SVM and kernel methods matlab toolbox. Perception Systmes et Information, INSA de Rouen, Rouen, France (2005)Google Scholar
  3. 3.
    Cao, L., Liu, Z., Huang, T.: Cross dataset action detection. In: CVPR. IEEE, Los Alamitos (2010)Google Scholar
  4. 4.
    Chen, L., Chin, K., Liao, H.: An integrated approach to video retrieval. In: ADC. Australian Computer Society, Inc. (2008)Google Scholar
  5. 5.
    Crucianu, M., Ferecatu, M., Boujemaa, N.: Relevance feedback for image retrieval: a short survey. State of the art in audiovisual content-based retrieval, information universal access and interaction including data models and languages, DELOS2 Report (FP6 NoE) (2004)Google Scholar
  6. 6.
    Duan, L., Xu, D., Tsang, I., Luo, J.: Visual event recognition in videos by learning from web data. In: CVPR. IEEE, Los Alamitos (2010)Google Scholar
  7. 7.
    Hauptmann, A., Lin, W., Yan, R., Yang, J., Chen, M.: Extreme video retrieval: joint maximization of human and computer performance. In: MULTIMEDIA. ACM, New York (2006)Google Scholar
  8. 8.
    Hu, Y., Cao, L., Lv, F., Yan, S., Gong, Y., Huang, T.: Action detection in complex scenes with spatial and temporal ambiguities. In: ICCV. IEEE, Los Alamitos (2009)Google Scholar
  9. 9.
    Laptev, I.: On space-time interest points. International Journal of Computer Vision 64, 107–123 (2005)CrossRefGoogle Scholar
  10. 10.
    Liu, J., Luo, J., Shah, M.: Recognizing realistic actions from videos “in the wild”. In: CVPR. IEEE, Los Alamitos (2009)Google Scholar
  11. 11.
    Liu, X., Zhuang, Y., Pan, Y.: A new approach to retrieve video by example video clip. In: MULTIMEDIA. ACM, New York (1999)Google Scholar
  12. 12.
    Liu, Y., Xu, D., Tsang, I., Luo, J.: Using large-scale web data to facilitate textual query based retrieval of consumer photos. In: MULTIMEDIA. ACM, New York (2009)Google Scholar
  13. 13.
    Luan, H., Zheng, Y., Neo, S., Zhang, Y., Lin, S., Chua, T.: Adaptive multiple feedback strategies for interactive video search. In: CIVR. ACM, New York (2008)Google Scholar
  14. 14.
    Pan, S., Yang, Q.: A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering (2009)Google Scholar
  15. 15.
    Rocchio, J.: Relevance Feedback in Information Retrieval, pp. 313–323. Prentice-Hall, Inc., Englewood Cliffs (1971)Google Scholar
  16. 16.
    Ruthven, I., Lalmas, M.: A survey on the use of relevance feedback for information access systems. Knowledge and Engineering Review 18, 95–145 (2003)CrossRefGoogle Scholar
  17. 17.
    Ryoo, M., Aggarwal, J.: Spatio-temporal relationship match: Video structure comparison for recognition of complex human activities. In: ICCV. IEEE, Los Alamitos (2009)Google Scholar
  18. 18.
    Settles, B.: Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin–Madison (2010)Google Scholar
  19. 19.
    Setz, A., Snoek, C.: Can social tagged images aid concept-based video search? In: ICME. IEEE, Los Alamitos (2009)Google Scholar
  20. 20.
    Tong, S., Chang, E.: Support vector machine active learning for image retrieval. In: MULTIMEDIA. ACM, New York (2001)Google Scholar
  21. 21.
    Yang, J., Yan, R., Hauptmann, A.: Cross-domain video concept detection using adaptive SVMs. In: MULTIMEDIA. ACM, New York (2007)Google Scholar
  22. 22.
    Yang, J., Hauptmann, A.: A framework for classifier adaptation and its applications in concept detection. In: MIR. ACM, New York (2008)Google Scholar
  23. 23.
    Yao, Y., Doretto, G.: Boosting for transfer learning with multiple sources. In: CVPR. IEEE, Los Alamitos (2010)Google Scholar
  24. 24.
    Zhou, X., Huang, T.: Relevance feedback in image retrieval: A comprehensive review. Multimedia Systems 8, 536–544 (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Antony Lam
    • 1
  • Amit K. Roy-Chowdhury
    • 2
  • Christian R. Shelton
    • 1
  1. 1.Dept. of Computer Science & EngineeringUniversity of CaliforniaRiversideUSA
  2. 2.Dept. of Electrical EngineeringUniversity of CaliforniaRiversideUSA

Personalised recommendations