A Framework for Combined Recognition of Actions and Objects
This paper proposes a novel approach to recognize actions and objects within the context of each other. Assuming that the different actions involve different objects in image sequences and there is one-to-one relation between object and action type, we present a Bayesian network based framework which combines motion patterns and object usage information to recognize actions/objects. More specifically, our approach recognizes high-level actions and the related objects without any body-part segmentation, hand tracking, and temporal segmentation methods. Additionally, we present a novel motion representation, based on 3D Haar-like features, which can be formed by depth, color, or both images. Our approach is also appropriate for object and action recognition where the involved object is partially or fully occluded. Finally, experiments show that our approach improves the accuracy of both action and object recognition significantly.
KeywordsAction and object recognition Bayesian Network motion pattern
Unable to display preview. Download preview PDF.
- 1.Ullman, S.: High-level vision: object recognition and visual cognition. MIT Press (1996)Google Scholar
- 2.Fei-Fei, L., Fergus, R., Torralba, A.: Recognizing and learning object categories: short course. In: ICCV (2009)Google Scholar
- 5.Filipovych, R., Ribeiro, E.: Recognizing primitive interactions by exploring actor-object states. In: CVPR (2008)Google Scholar
- 6.Wu, J., Osuntogun, A., Choudhury, T., Philipose, M., Rehg, J.M.: A scalable approach to activity recognition based on object use. In: ICCV, pp. 1–7 (2007)Google Scholar
- 10.Gall, J., Fossati, A., Gool, L.J.V.: Functional categorization of objects using real-time markerless motion capture. In: CVPR, pp. 1969–1976 (2011)Google Scholar
- 11.Ciu, X., Liu, Y., Shan, S., Chen, X., Gao, W.: 3D Haar-like features for pedestrian detection. In: ICME 2007, pp. 1263–1266 (2007)Google Scholar
- 12.Fei-Fei, L.: Bag of words models: recognizing and learning object categories. In: CVPR 2007 (2007)Google Scholar
- 13.Rubinstein, Y., Hastie, T.: Discriminative vs. informative learning. In: Proc. of the 3th Int. Conf. on Knowledge Discovery and Data Mining, pp. 49–53 (1997)Google Scholar