Learning Actions Using Robust String Kernels

Abstract

This paper presents an action analysis method based on robust string matching using dynamic programming. Similar to matching text sequences, atomic actions based on semantic and structural features are first detected and coded as spatio-temporal characters or symbols. These symbols are subsequently concatenated to form a unique set of strings for each action. A similarity metric using longest common subsequence algorithm is employed to robustly match action strings with variable length. A dynamic programming method with polynomial computational complexity and linear space complexity is implemented. An effective learning scheme based on similarity metric embedding is developed to deal with matching strings of variable length. Our proposed method works with limited amount of training data and exhibits desirable generalization property. Moreover, it can be naturally extended to detect compound behaviors and events. Experimental evaluation on our own and a commonly used data set demonstrates that our method allows for large pose and appearance changes, is robust to background clutter, and can accommodate spatio-temporal behavior variations amongst different subjects while achieving high discriminability between different behaviors.