Abstract
This report establishes a novel concept for tracking complex and articulated objects in the presence of high observation uncertainties utilising Markov random fields Markov chains (MRFMCs) and a novel paradigm of modelling visual perception. The approach is rooted in ideas from information fusion and cognitive sciences. The problem is to track non-rigid and articulated objects in the 3D space. The aim is to precisely estimate landmarks with high certainty for fitting accurate object models and secondary states like the orientation under partial occlusions. The targeted system is characterised by a high degree of generality. Previous solutions are relatively limited in robustness and accuracy. The new concept is motivated by the fact that all previous tracking approaches rely on semantic information, that is classified signal signatures, while neglecting all further non-classifiable and thus semantically unrelated information present in the scene herein abstracted as structure. By observing salient cues in structure and by learning and incorporating topological relations between salient cues and semantic features it is intended to tackle the major problem of visual tracking, namely accurate and robust inference in the presence of high observation uncertainties. The notion of the dichotomy of semantic and structure is not covered in previous literature. The new concept constitutes a novel direction in the design and implementation of visual perception and tracking networks. While the ideas of dynamic world modelling and intelligent forgetting stem from principles of information fusion, the principle of fusing semantical with structural information from intelligent exploring is an entirely original contribution and is inspired by ideas from cognitive sciences and linguistics. It is deduced from the inherent yet unrevealed principle of appearance modelling, which is based on incorporating object-related appearance information without classification. In this report the presented system is applied to high-level facial pose tracking and compared to a state-of-the-art reference method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Agnew, D., Constable, C.: Geophysical data analysis: Multivariate random variables, correlation and error propagation (2008)
Allen, P.K.: 3d photography: Point based rigid registration. Technical report, Columbia Department of Computer Science, Columbia University (2005)
Arun, K.S., Huang, T.S., Blostein, S.D.: Least-squares fitting of two 3-d point sets. IEEE Trans. PAMI 9, 698–700 (1987)
Bishop, C.M.: A new framework for machine learning. In: Zurada, J.M., Yen, G.G., Wang, J. (eds.) Computational Intelligence: Research Frontiers. LNCS, vol. 5050, pp. 1–24. Springer, Heidelberg (2008)
Bishop, C.M.: Pattern Recognition and Machine Learning: Graphical Models. Springer, Heidelberg (2006)
Blake, A., Isard, M.: Active Contours. Springer, Heidelberg (1997)
Chang, W.-Y., Chen, C.-S., Hung, Y.-P.: Tracking by parts: A bayesian approach with component collaboration. IEEE Transactions on Systems, Man, and Cybernetics 39, 375–388 (2009)
Constable, C., Agnew, D.C.: Geophysical data analysis: Statistics (2005)
Crowley, J.L., Demazeau, Y.: Principle and techniques for sensor data fusion. Signal Processing 32, 5–27 (1993)
Del Bue, A., Agapito, L.: Non-rigid 3d shape recovery using stereo factorization. In: Asian Conference of Computer Vision (ACCV), vol. 1, pp. 25–30 (2004)
Del Bue, A., Smeraldi, F., Agapito, L.: Non-rigid structure from motion using ranklet-based tracking and non-linear optimization. IVC 25(3), 297–310 (2007)
Doucet, A., Johansen, A.M.: A tutorial on particle filtering and smoothing: Fiteen years later (2009)
Du, W., Piater, J.: A probabilistic approach to integrating multiple cues in visual tracking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 225–238. Springer, Heidelberg (2008)
Gales, M.J.F., Airey, S.S.: Product of gaussians for speech recognition. In: Computer Speech and Language (2006)
Hall, D.: Mathematical Techniques in Multisensor Data Fusion. Artech House, Boston (1992)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2004)
Haug, A.J.: A tutorial on bayesian estimation and tracking techniques applicable to nonlinear and non-gaussian processes. Technical report, The MITRE Corporation (2005)
Isard, M.: Pampas: Real-valued graphical models for computer vision. Technical report, Microsoft Research (2003)
Isard, M., Blake, A.: Condensation - conditional density propagation for visual tracking (1998)
Johnson, J.K.: Estimation of gmrfs by recursive cavity modeling. Technical report, EECS Dept., MIT (2004)
Jordan, M.I., Weiss, Y.: The Handbook of Brain Theory and Neural Networks. Graphical models: Probabilistic inference. MIT Press, Cambridge (2002)
Jordan, M.I.: An introduction to probabilistic graphical models. Technical report, University of California, Berkeley (2003)
Kropatsch, W.: Tracking with structure in computer vision twist-cv. Technical report, Patter Recognition and Image Processing Group, TU Wien (2005)
Li, T., Kallem, V., Singaraju, D., Vidal, R.: Projective factorization of multiple rigid-body motions. In: CVPR 2007, pp. 1–6 (2007)
Malioutov, D.M.: Approximate Inference in Gaussian Graphical Models. PhD thesis, Dept. of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge (2008)
Malioutov, D.M., Johnson, J.K., Willsky, A.S.: Walk-sums and belief propagation in gaussian graphical models. Journal of Machine Learning Research 7, 2031–2064 (2006)
Mills, S.: Stereo-motion analysis of image sequences. In: Proceedings of Digital Image & Vision Computing: Techniques and Applications (DICTA), pp. 515–520 (1997)
Mills, S., Novins, K.: Graph-based object hypothesis. New Zealand Journal of Computing 7, 21–29 (1998)
Mills, S., Novins, K.: Motion segmentation in long image sequences. In: Proceedings of the British Machine Vision Conference, pp. 162–171 (2000)
Murphy, K.P.: An introduction to graphical models. Technical report, University of British Columbia, Vancouver, Canada (2001)
Newman, P., Leonard, J.: A matrix oriented note on joint, marginal, and conditional multivariate gaussian distributions. Technical report, Massachusetts Institute of Technology (2006)
Ristic, B., Arulampalam, S., Gordon, N.: Beyond the Kalman Filter: Particle Filters for Tracking Applications. Artech House Publishers, Boston (2004)
Rong Li, X., Jilkov, V.P.: Survey of maneuvering target tracking. part i: Dynamic models. IEEE Transactions on Aerospace and Electronic Systems 39, 1333–1364 (2003)
Sanfeliu, A., Serratosa, F.: Learning and recognising 3d models represented by multiple views by means of methods based on random graphs. In: Proceedings International Conference on Image Processing, ICIP (2003)
Sigal, L., Zhu, Y., Comaniciu, D., Black, M.J.: Tracking complex objects using graphical object models. In: Jähne, B., Mester, R., Barth, E., Scharr, H. (eds.) IWCM 2004. LNCS, vol. 3417, pp. 223–234. Springer, Heidelberg (2007)
Sinha, A., Chen, H., Danu, D.G., Kirubarajan, T., Farooq, M.: Estimation and decision fusion: A survey. Neurocomputing 71, 2650–2656 (2008)
Smith, D., Singh, S.: Approaches to multisensor data fusion in target tracking: A survey. IEEE Transactions on Knowledge and Data Engineering 18(12), 1696–1710 (2006)
Steffens, M., Krybus, W., Kohring, C.: Linear gaussian error models from component matrices for 3d graphical tracking networks. In: Submitted to SAMT 2010 (2010)
Steffens, M., Krybus, W., Kohring, C.: Spatio-temporal gaussian graphical models as tracking networks. In: Submitted to SAMT 2010 (2010)
Su, C., Huang, L.: Spatio-temporal graphical-model-based multiple facial feature tracking. EURASIP Journal on Applied Signal Processing 13, 2091–2100 (2005)
Sudderth, E.B., Ihler, A.T., Freeman, W.T., Willsky, A.S.: Nonparametric belief propagation. In: Conference Proceedings Computer Vision and Pattern Recognition. IEEE, Los Alamitos (2003)
Tang, C.-Y., Hung, Y.-P., Shih, S.-W., Chen, Z.: A 3d feature-based tracker for multiple object tracking. In: Proceedings of the National Science Council, Republic of China, Part A: Physical Science and Engineering, pp. 151–168 (1999)
Taycher, L., Fisher III, J.W., Darrell, T.: Combining object and feature dynamics in probabilistic tracking. Computer Vision and Image Understanding 108, 243–260 (2007)
Vidal, R., Abretske, D.: Nonrigid shape and motion from multiple perspective views. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 205–218. Springer, Heidelberg (2006)
Vidal, R., Ma, Y.: A unified algebraic approach to 2-d and 3-d motion segmentation. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 1–15. Springer, Heidelberg (2004)
Vidal, R., Ma, Y., Soatto, S., Sastry, S.: Two-view multibody structure from motion. IJCV 68(1), 7–25 (2006)
Vidal, R., Ravichandran, A.: Optical flow estimation and segmentation of multiple moving dynamic textures. In: CVPR 2005, pp. II: 516–521 (2005)
Vidal, R., Singaraju, D.: A closed form solution to direct motion segmentation. In: CVPR 2005, pp. II: 510–515 (2005)
Wang, P., Ji, Q.: Robust face tracking via collaboration of generic and specific models. IEEE Transactions on Image Processing 17, 1189–1199 (2008)
Weingarten, J.W., Gruener, G., Siegwart, R.: Probabilistic plane fitting in 3d and an application to robotic mapping. In: IEEE International Conference on Robotics and Automation (2004)
Yang, M., Wu, Y.: Granularity and elasticity adaptation in visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2008)
Yedidia, J.S., Freeman, W.T., Weiss, Y.: Understanding belief propagation and its generalizations. Technical report. MIT, Cambridge (2001)
Yu, T., Wu, Y.: Decentralized multiple target tracking using netted collaborative autonomous trackers. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 939–946 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Steffens, M., Krybus, W., Kohring, C. (2011). Dynamic World Modelling by Dichotomic Information Sets and Graphical Inference. In: Declerck, T., Granitzer, M., Grzegorzek, M., Romanelli, M., Rüger, S., Sintek, M. (eds) Semantic Multimedia. SAMT 2010. Lecture Notes in Computer Science, vol 6725. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23017-2_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-23017-2_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23016-5
Online ISBN: 978-3-642-23017-2
eBook Packages: Computer ScienceComputer Science (R0)