Learning to Predict Where People Look with Tensor-Based Multi-view Learning

  • Kitsuchart PasupaEmail author
  • Sandor Szedmak
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9489)


Eye movements data collection is very expensive and laborious. Moreover, there are usually missing values. Assuming that we are collecting eye movements data on a set of images from different users (views). There is a possibility that we are not able to collect eye movements of all users on all images. One or more views are not represented in the image. We assume that the relationships among the views can be learnt from the complete items. The task is then to reproduce the missing part of the incomplete items from the relationships derived from the complete items and the known part of these items. Using the properties of tensor algebra we show that this problem can be formulated consistently as a regression type learning task. Furthermore, there is a maximum margin based optimisation framework where this problem can be solved in a tractable way. This problem is similar to learning to predict where human look. The proposed algorithm is proved to be more effective than well-known saliency detection techniques.


Multi-view learning Missing data Tensor algebra One rank tensor approximation Maximum margin learning Eye movements 


  1. 1.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998)CrossRefGoogle Scholar
  2. 2.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems, pp. 545–552 (2006)Google Scholar
  3. 3.
    Judd, T., Ehinger, K., Durand, F., Torralba, A.: Learning to predict where humans look. In: IEEE 12th International Conference on Computer Vision, pp. 2106–2113 (2009)Google Scholar
  4. 4.
    Henderson, J.M., Brockmole, J.R., Castelhano, M.S., Mack, M.: Visual saliency does not account for eye movements during visual search in real-world scenes. In: Eye Movements: A Window on Mind and Brain, pp. 537–562 (2007)Google Scholar
  5. 5.
    Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. IEEE Trans. Pattern Anal. Mach. Intell. 35, 208–220 (2013)CrossRefGoogle Scholar
  6. 6.
    Chen, C.Y., Grauman, K.: Inferring unseen views of people. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2011–2018 (2014)Google Scholar
  7. 7.
    Itskov, M.: Tensor Algebra and Tensor Analysis for Engineers With Applications to Continuum Mechanics. 2nd edn. Springer, Heidelberg (2009)Google Scholar
  8. 8.
    Synge, J., Schild, A.: Tensor Calculus. Dover, New York (1978)zbMATHGoogle Scholar
  9. 9.
    Astikainen, K., Holm, L., Pitkänen, E., Szedmak, S., Rousu, J.: Towards structured output prediction of enzyme function. In: BMC Proceedings, vol. 2(Suppl 4:S2) (2008)Google Scholar
  10. 10.
    Szedmak, S., De Bie, T., Hardoon, D.R.: A metamorphosis of canonical correlation analysis into multivariate maximum margin learning. In: The 15th European Symposium on Artificial Neural Networks, pp. 211–216 (2007)Google Scholar
  11. 11.
    Briët, J., Harremoës, P.: Properties of classical and quantum Jensen-Shannon divergence. Phys. Rev. A 79, 052311 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Faculty of Information TechnologyKing Mongkut’s Institute of Technology LadkrabangBangkokThailand
  2. 2.Institute of Computer ScienceUniversity of InnsbruckInnsbruckAustria

Personalised recommendations