Abstract
We present a novel, low-level scheme to analyze spatial and temporal change within a local support region. Assuming available region correspondences between two adjacent frames, we divide each region into a regular grid of patches. Depending on the change of an image function inside the patch over time, each patch is assigned weights for the following four labels: “C” for a constant patch, “O” when new information originates from outside the support region, “I” for “inner” changes, and “N” for information from neighboring patches. Our method goes beyond optical flow, as it provides an additional semantic level of understanding the changes in space-time. We demonstrate how our novel “COIN” scheme can be used to categorize local space-time events in image pairs, including locally planar support regions, 3D discontinuities, and virtual vs. real crossings of 3D structures.
Chapter PDF
References
Doretto, G., Chiuso, A., Wu, Y., Soatto, S.: Dynamic textures. Int. J. of Comp. Vis. 51, 91–109 (2003)
Derpanis, K., Wildes, R.: Spacetime texture representation and recognition based on a spatiotemporal orientation analysis. IEEE TPAMI (2012)
Doulamis, A., Doulamis, N., Ntalianis, K., Kollias, S.: An efficient fully unsupervised video object segmentation scheme using an adaptive neural-network classifier architecture. Trans. Neur. Netw. 14, 616–630 (2003)
Ogale, A.S., Fermüller, C., Aloimonos, Y.: Motion segmentation using occlusions. IEEE Trans. Pattern Anal. Mach. Intell. 27, 988–992 (2005)
Stein, A.N., Hebert, M.: Local detection of occlusion boundaries in video. Image Vision Comput. 27, 514–522 (2009)
Derpanis, K.G., Wildes, R.P.: Detecting Spatiotemporal Structure Boundaries: Beyond Motion Discontinuities. In: Zha, H., Taniguchi, R.-I., Maybank, S. (eds.) ACCV 2009, Part II. LNCS, vol. 5995, pp. 301–312. Springer, Heidelberg (2010)
Dollár, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: VS-PETS (2005)
Ke, Y., Sukthankar, R., Hebert, M.: Efficient visual event detection using volumetric features. In: Proc. ICCV, vol. 1, pp. 166–173 (2005)
Laptev, I., Lindeberg, T.: Space-time interest points. In: Proc. ICCV (2003)
Laptev, I.: On space-time interest points. Int. J. of Comp. Vis. 64, 107–123 (2005)
Le, Q.V., Zou, W.Y., Yeung, S.Y., Ng, A.Y.: Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis. In: Proc. CVPR, pp. 3361–3368 (2011)
Wang, H., Ullah, M.M., Kläser, A., Laptev, I., Schmid, C.: Evaluation of local spatio-temporal features for action recognition. In: Proc. BMVC, p. 127 (2009)
Kläser, A., Marszałek, M., Schmid, C.: A spatio-temporal descriptor based on 3d-gradients. In: Proc. BMVC, pp. 995–1004 (2008)
Brkić, K., Pinz, A., Šegvić, S., Kalafatić, Z.: Histogram-Based Description of Local Space-Time Appearance. In: Heyden, A., Kahl, F. (eds.) SCIA 2011. LNCS, vol. 6688, pp. 206–217. Springer, Heidelberg (2011)
Aanæs, H., Dahl, A., Steenstrup Pedersen, K.: Interesting interest points. Int. J. of Comp. Vis. 97, 18–35 (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Brkić, K., Pinz, A., Kalafatić, Z., Šegvić, S. (2012). Towards Space-Time Semantics in Two Frames. In: Fusiello, A., Murino, V., Cucchiara, R. (eds) Computer Vision – ECCV 2012. Workshops and Demonstrations. ECCV 2012. Lecture Notes in Computer Science, vol 7585. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33885-4_13
Download citation
DOI: https://doi.org/10.1007/978-3-642-33885-4_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33884-7
Online ISBN: 978-3-642-33885-4
eBook Packages: Computer ScienceComputer Science (R0)