Open-view human action recognition based on linear discriminant analysis



In the last decades, action recognition task has evolved from single view recording to unconstrained environment. Recently, multi-view action recognition has become a hot topic in computer vision. However, we notice that only a few works have focused on the open-view action recognition, which is a common problem in the real world. Open-view action recognition focus on doing action recognition in unseen view without using any information from it. To address this issue, we firstly introduce a novel multi-view surveillance action dataset and benchmark several state-of-the-art algorithms. From the results, we observe that the performance of the state-of-the-art algorithms would drop a lot under open-view constraints. Then, we propose a novel open-view action recognition method based on the linear discriminant analysis. This method can learn a common space for action samples under different view by using their category information, which can achieve a good result in open-view action recognition.


Action recognition Open-view Dataset 



This work was supported in part by the National Natural Science Foundation of China under Grant 61772359 and Grant 61472275 and Grant 61572356 and in part by the Tianjin Research Program of Application Foundation and Advanced Technology under Grant 15JCYBJC16200.


  1. 1.
    Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54 (11):4311–4322CrossRefMATHGoogle Scholar
  2. 2.
    Arthur D, Vassilvitskii S (2007) k-means++: The advantages of careful seeding. In: ACM-SIAM symposium on discrete algorithms, pp 1027–1035Google Scholar
  3. 3.
    Cai D, He X, Han J (2008) Srda: an efficient algorithm for large-scale discriminant analysis. IEEE Trans Knowl Data Eng 20(1):1–12CrossRefGoogle Scholar
  4. 4.
    Chang X, Yang Y (2016) Semisupervised feature analysis by mining correlations among multiple tasks. IEEE Trans Neural Netw Learn Syst PP(99):1–12Google Scholar
  5. 5.
    Chang X, Yu YL, Yang Y, Xing EP (2016) They are not equally reliable: Semantic event search using differentiated concept classifiers. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1884–1893Google Scholar
  6. 6.
    Dollár P, Rabaud V, Cottrell G, Belongie S (2005) Behavior recognition via sparse spatio-temporal features. In: IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp 65–72Google Scholar
  7. 7.
    Fan RE, Chang KW, Hsieh CJ, Wang XR, Lin CJ (2008) Liblinear: a library for large linear classification. J Mach Learn Res 9:1871–1874MATHGoogle Scholar
  8. 8.
    Faraki M, Palhang M, Sanderson C (2014) Log-euclidean bag of words for human action recognition. IET Comput Vis 9(3):331–339CrossRefGoogle Scholar
  9. 9.
    Gorelick L, Blank M, Shechtman E, Irani M, Basri R (2007) Actions as space-time shapes. IEEE Trans Pattern Anal Mach Intell 29(12):2247–2253CrossRefGoogle Scholar
  10. 10.
    Hao T, Peng W, Wang Q, Wang B, Sun J (2016a) Reconstruction and application of proteincprotein interaction network. Int J Mol Sci 17(6):907CrossRefGoogle Scholar
  11. 11.
    Hao T, Yu AL, Peng W, Wang B, Sun JS (2016b) Cross domain mitotic cell recognition. Neurocomputing 195(C):6–12CrossRefGoogle Scholar
  12. 12.
    He X, Gao M, Kan MY, Wang D (2017) Birank: towards ranking on bipartite graphs. IEEE Trans Knowl Data Eng 29(1):57–71CrossRefGoogle Scholar
  13. 13.
    Ke Y, Sukthankar R, Hebert M (2007) Event detection in crowded videos. In: IEEE International Conference on Computer Vision, pp 1–8Google Scholar
  14. 14.
    Kliper-Gross O, Hassner T, Wolf L (2012) The action similarity labeling challenge. IEEE Trans Pattern Anal Mach Intell 34(3):615–621CrossRefGoogle Scholar
  15. 15.
    Kuehne H, Jhuang H, Garrote E, Poggio T, Serre T (2011) HMDB: a large video database for human motion recognition. In: IEEE International Conference on Computer Vision, pp 2556–2563Google Scholar
  16. 16.
    Kuehne H, Arslan A, Serre T (2014) The language of actions: Recovering the syntax and semantics of goal-directed human activities. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 780–787Google Scholar
  17. 17.
    Laptev I, Lindeberg T (2003) Space-time interest points. In: IEEE International Conference on Computer Vision, pp 432–439Google Scholar
  18. 18.
    Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1–8Google Scholar
  19. 19.
    Liu A, Su Y, Jia PP, Gao Z, Hao T, Yang ZX (2015) Multipe/single-view human action recognition via part-induced multitask structural learning. IEEE Trans Cybern 45(6):1194–1208CrossRefGoogle Scholar
  20. 20.
    Liu A, Nie W, Gao Y, Su Y (2016a) Multi-modal clique-graph matching for view-based 3d model retrieval. IEEE Trans Image Process 25(5):2103–2116MathSciNetCrossRefGoogle Scholar
  21. 21.
    Liu A, Ning X, Nie W, Su Y, Wong Y, Kankanhalli M (2016b) Benchmarking a multimodal and multiview and interactive dataset for human action recognition. IEEE Trans Cybern PP(99):1–14Google Scholar
  22. 22.
    Liu A, Su Y, Nie W, Kankanhalli M (2016c) Hierarchical clustering multi-task learning for joint human action grouping and recognition. IEEE Trans Pattern Anal Mach Intell 39(1):102–114CrossRefGoogle Scholar
  23. 23.
    Liu J, Luo J, Shah M (2009) Recognizing realistic actions from videos in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1996–2003Google Scholar
  24. 24.
    Luo M, Chang X, Nie L, Yang Y, Hauptmann AG, Zheng Q (2017) An adaptive semisupervised feature analysis for video semantic recognition. IEEE Trans Cybern PP(99):1–13CrossRefGoogle Scholar
  25. 25.
    Marszalek M, Laptev I, Schmid C (2009) Actions in context. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2929–2936Google Scholar
  26. 26.
    Messing R, Pal C, Kautz H (2009) Activity recognition using the velocity histories of tracked keypoints. In: IEEE International Conference on Computer Vision, pp 104–111Google Scholar
  27. 27.
    Nie L, Wang M, Zha ZJ, Chua TS (2012) Oracle in image search: a content-based approach to performance prediction. ACM Trans Inf Syst 30(2):13CrossRefGoogle Scholar
  28. 28.
    Nie L, Zhang L, Meng L, Song X, Chang X, Li X (2016) Modeling disease progression via multisource multitask learners: a case study with alzheimer’s disease. In: IEEE Transactions on Neural Networks and Learning Systems, pp 1–12Google Scholar
  29. 29.
    Nie W, Liu A, Li W, Su Y (2016b) Cross-view action recognition by cross-domain learning. Image Vis Comput 55:109–118CrossRefGoogle Scholar
  30. 30.
    Niebles JC, Chen C, Li F (2010) Modeling temporal structure of decomposable motion segments for activity classification. Lect Notes Comput Sci 6312:392–405CrossRefGoogle Scholar
  31. 31.
    Over P, Fiscus J, Sanders G, Joy D, Michel M, Awad G, Smeaton A, Kraaij W, Quénot G (2014) TRECVID 2014 – an overview of the goals, tasks, data, evaluation mechanisms and metrics. In: Proceedings of TRECVID, p 52Google Scholar
  32. 32.
    Rodriguez MD, Ahmed J, Shah M (2008) Action mach a spatio-temporal maximum average correlation height filter for action recognition. In: CVPR 2008 IEEE Conference on Computer Vision and pattern Recognition, pp 1–8Google Scholar
  33. 33.
    Ryoo MS, Aggarwal JK (2009) Spatio-temporal relationship match: video structure comparison for recognition of complex human activities. In: IEEE International Conference on Computer Vision, pp 1593–1600Google Scholar
  34. 34.
    Schuldt C, Laptev I, Caputo B (2004) Recognizing human actions: a local SVM approach. In: International Conference on Pattern Recognition, pp 32–36Google Scholar
  35. 35.
    Sigal L, Balan AO, Black MJ (2010) Humaneva: synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. Int J Comput Vis 87(1):4CrossRefGoogle Scholar
  36. 36.
    Singh S, Velastin SA, Ragheb H (2010) Muhavi: a multicamera human action video dataset for the evaluation of action recognition methods. In: IEEE International Conference on Advanced Video and Signal Based Surveillance, pp 48–55Google Scholar
  37. 37.
    Soomro K, Zamir AR, Shah M (2012) Ucf101: a dataset of 101 human actions classes from videos in the wild. Computer ScienceGoogle Scholar
  38. 38.
    Tropp JA, Gilbert AC (2007) Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans Inf Theory 53(12):4655–4666MathSciNetCrossRefMATHGoogle Scholar
  39. 39.
    UCF (2011) Aerial action dataset.
  40. 40.
    UCF (2011) Aerial camera, rooftop camera and ground camera dataset.
  41. 41.
    Wang H, Schmid C (2013) Action recognition with improved trajectories. In: IEEE International Conference on Computer Vision, pp 3551–3558Google Scholar
  42. 42.
    Wang H, Ullah MM, Klaser A, Laptev I, Schmid C (2009) Evaluation of local spatio-temporal features for action recognition. In: British Machine Vision Conference, pp 124–1Google Scholar
  43. 43.
    Wang H, Kläser A, Schmid C, Liu CL (2013) Dense trajectories and motion boundary descriptors for action recognition. Int J Comput Vis 103(1):60–79MathSciNetCrossRefGoogle Scholar
  44. 44.
    Weinland D, Boyer E, Ronfard R (2007) Action recognition from arbitrary views using 3d exemplars. In: IEEE International Conference on Computer Vision, pp 1–7Google Scholar
  45. 45.
    Yao B, Jiang X, Khosla A, Lin AL, Guibas L, Fei-Fei L (2011) Human action recognition by learning bases of action attributes and parts. In: International Conference on Computer Vision, pp 1331– 1338Google Scholar
  46. 46.
    Zhang H, Shang X, Luan H, Wang M, Chua TS (2016) Learning from collective intelligence: Feature learning using social images and tags. ACM Trans Multimed Comput Commun Appl 13(1):1CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Electrical and Information EngineeringTianjin UniversityTianjinChina

Personalised recommendations