Advertisement

Learning Actions Using Robust String Kernels

  • Changjiang Yang
  • Yanlin Guo
  • Harpreet S. Sawhney
  • Rakesh Kumar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4814)

Abstract

This paper presents an action analysis method based on robust string matching using dynamic programming. Similar to matching text sequences, atomic actions based on semantic and structural features are first detected and coded as spatio-temporal characters or symbols. These symbols are subsequently concatenated to form a unique set of strings for each action. A similarity metric using longest common subsequence algorithm is employed to robustly match action strings with variable length. A dynamic programming method with polynomial computational complexity and linear space complexity is implemented. An effective learning scheme based on similarity metric embedding is developed to deal with matching strings of variable length. Our proposed method works with limited amount of training data and exhibits desirable generalization property. Moreover, it can be naturally extended to detect compound behaviors and events. Experimental evaluation on our own and a commonly used data set demonstrates that our method allows for large pose and appearance changes, is robust to background clutter, and can accommodate spatio-temporal behavior variations amongst different subjects while achieving high discriminability between different behaviors.

Keywords

Dynamic Programming Action Recognition Interest Point Atomic Action String Match 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Efros, A.A., Berg, A.C., Mori, G., Malik, J.: Recognizing action at a distance. In: International Conference on Computer Vision, Nice, France, pp. 726–733 (2003)Google Scholar
  2. 2.
    Ke, Y., Sukthankar, R., Hebert, M.: Efficient visual event detection using volumetric features. In: International Conference on Computer Vision, vol. I, pp. 166–173 (October 2005)Google Scholar
  3. 3.
    Schuldt, C., Laptev, I., Caputo, B.: Recognizing human actions: A local svm approach. In: International Conference on Pattern Recognition, Cambridge, United Kingdom, vol. 3, pp. 32–36 (August 2004)Google Scholar
  4. 4.
    Zhong, H., Shi, J., Visontai, M.: Detecting unusual activity in video. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. II, pp. 819–826. IEEE Computer Society Press, Los Alamitos (2004)Google Scholar
  5. 5.
    Boiman, O., Irani, M.: Similarity by composition. In: Neural Information Processing Systems, Vancouver, Canada (2006)Google Scholar
  6. 6.
    Yacoob, Y., Black, M.: Parameterized modeling and recognition of activities. Computer Vision and Image Understanding (CVIU) 73, 232–247 (1999)CrossRefGoogle Scholar
  7. 7.
    Davis, J.W., Bobick, A.F.: The representation and recognition of action using temporal templates. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 928–934. IEEE Computer Society Press, Los Alamitos (1997)Google Scholar
  8. 8.
    Sivic, J., Zisserman, A.: Video Google: A text retrieval approach to object matching in videos. In: International Conference on Computer Vision, Nice, France, vol. 2, pp. 1470–1477 (October 2003)Google Scholar
  9. 9.
    Schodl, A., Szeliski, R., Salesin, D.H., Essa, I.: Video textures. In: Proceedings of the conference on Computer graphics and interactive techniques, pp. 489–498 (2000)Google Scholar
  10. 10.
    Zelnik-Manor, L., Irani, M.: Event-based analysis of video. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. II, pp. 123–130. IEEE Computer Society Press, Los Alamitos (2001)Google Scholar
  11. 11.
    Black, M.J., Jepson, A.D.: Eigentracking: Robust matching and tracking of articulated objects using view-based representation. International Journal of Computer Vision 26(1), 63–84 (1998)CrossRefGoogle Scholar
  12. 12.
    Shechtman, E., Irani, M.: Space-time behavior based correlation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 405–412. IEEE Computer Society Press, Los Alamitos (2005)Google Scholar
  13. 13.
    Laptev, I., Lindeberg, T.: Space-time interest points. In: International Conference on Computer Vision, pp. 432–439 (2003)Google Scholar
  14. 14.
    Viola, P., Jones, M.: Robust real-time object detection. International Journal of Computer Vision 57(2), 137–154 (2004)CrossRefGoogle Scholar
  15. 15.
    Gusfield, D.: Algorithms on Strings, Trees and Sequences–Computer Science and Computational Biology. Cambridge University Press, Cambridge (1997)zbMATHGoogle Scholar
  16. 16.
    Lodhi, H., Saunders, C., Shawe-Taylor, J., Cristianini, N., Watkins, C.: Text classification using string kernels. J. Mach. Learn. Res. 2, 419–444 (2002)zbMATHCrossRefGoogle Scholar
  17. 17.
    Leslie, C.S., Eskin, E., Cohen, A., Weston, J., Noble, W.S.: Mismatch string kernels for discriminative protein classification. Bioinformatics 20(4), 467–476 (2004)CrossRefGoogle Scholar
  18. 18.
    Ivanov, Y., Bobick, A.: Recognition of visual activities and interactions by stochastic parsing. IEEE Trans. on Pattern Analysis and Machine Intelligence 22, 852–872 (2000)CrossRefGoogle Scholar
  19. 19.
    Dollar, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: ICCV VS-PETS, Beijing, China, pp. 65–72 (2005)Google Scholar
  20. 20.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 2nd edn. MIT Press and McGraw-Hill (2001)Google Scholar
  21. 21.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  22. 22.
    Shi, J., Tomasi, C.: Good features to track. In: IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, pp. 593–600. IEEE Computer Society Press, Los Alamitos (1994)Google Scholar
  23. 23.
    Ojala, T., Pietikainen, M., Harwood, D.: A comparative study of texture measures with classification based on feature distributions. Pattern Recognition 29, 51–59 (1996)CrossRefGoogle Scholar
  24. 24.
    Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 801, pp. 151–158. Springer, Heidelberg (1994)CrossRefGoogle Scholar
  25. 25.
    Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)zbMATHGoogle Scholar
  26. 26.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Changjiang Yang
    • 1
  • Yanlin Guo
    • 1
  • Harpreet S. Sawhney
    • 1
  • Rakesh Kumar
    • 1
  1. 1.Sarnoff Corporation, Princeton, NJ 08543 

Personalised recommendations