Mixed-State Markov Models in Image Motion Analysis

  • Tomás Crivelli
  • Patrick Bouthemy
  • Bruno Cernuschi Frías
  • Jian-feng Yao
Part of the Advances in Pattern Recognition book series (ACVPR)


When analyzing motion observations extracted from image sequences, one notes that the histogram of the velocity magnitude at each pixel shows a large probability mass at zero velocity, while the rest of the motion values may be appropriately modeled with a continuous distribution. This suggests the introduction of mixed-state random variables that have probability mass concentrated in discrete states, while they have a probability density over a continuous range of values. In the first part of the chapter, we give a comprehensive description of the theory behind mixed-state statistical models, in particular the development of mixed-state Markov models that permits to take into account spatial and temporal interaction. The presentation generalizes the case of simultaneous modeling of continuous values and any type of discrete symbolic states. For the second part, we present the application of mixed-state models to motion texture analysis. Motion textures correspond to the instantaneous apparent motion maps extracted from dynamic textures. They depict mixed-state motion values with a discrete state at zero and a Gaussian distribution for the rest. Mixed-state Markov random fields and mixed-state Markov chains are defined and applied to motion texture recognition and tracking.


Random Field Markov Random Field Discrete State Gibbs Distribution Leibler Divergence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Anderson, B., Moore, J.: Optimal filtering. Englewood Cliffs, Prentice-Hall (1979) MATHGoogle Scholar
  2. 2.
    Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994) CrossRefGoogle Scholar
  3. 3.
    Besag, J.: Spatial interaction and the statistical analysis of lattice systems. J. R. Stat. Soc., Ser. B 36, 192–236 (1974) MathSciNetMATHGoogle Scholar
  4. 4.
    Besag, J.: On the statistical analysis of dirty pictures. J. R. Stat. Soc., Ser. B 48(3), 259–302 (1986) MathSciNetMATHGoogle Scholar
  5. 5.
    Black, M., Rangarajan, A.: On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. Int. J. Comput. Vis. 19(1), 57–91 (1996) CrossRefGoogle Scholar
  6. 6.
    Bouthemy, P., Hardouin, C., Piriou, G., Yao, J.F.: Mixed-state auto-models and motion texture modeling. J. Math. Imaging Vis. 25(3), 387–402 (2006) MathSciNetCrossRefGoogle Scholar
  7. 7.
    Bruhn, A., Weickert, J., Feddern, C., Kohlberger, T., Schnorr, C.: Variational optical flow computation in real time. IEEE Trans. Image Process. 14(5), 608–615 (2005) MathSciNetCrossRefGoogle Scholar
  8. 8.
    Cernuschi-Frias, B.: Mixed-states Markov random fields with symbolic labels and multidimensional real values. Rapport de Recherche, INRIA No 6255 (2007).
  9. 9.
    Chan, A., Vasconcelos, N.: Modeling, clustering, and segmenting video with mixtures of dynamic textures. IEEE Trans. Pattern Anal. Mach. Intell. 30(5), 909–926 (2008) CrossRefGoogle Scholar
  10. 10.
    Chan, A.B., Vasconcelos, N.: Layered dynamic textures. IEEE Trans. Pattern Anal. Mach. Intell. 31(10), 1862–1879 (2009). CrossRefGoogle Scholar
  11. 11.
    Chetverikov, D., Peteri, R.: A brief survey of dynamic texture description and recognition. In: Proc. 4th International Conference on Computer Recognition Systems, CORES’05, Advances in Soft Computing, pp. 17–26. Springer, Berlin (2005) Google Scholar
  12. 12.
    Corpetti, T., Memin, E., Perez, P.: Dense estimation of fluid flows. IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 365–380 (2002) CrossRefGoogle Scholar
  13. 13.
    Cover, T., Thomas, J.: Elements of Information Theory. Wiley, New York (1991) MATHCrossRefGoogle Scholar
  14. 14.
    Crivelli, T., Cernuschi, B., Bouthemy, P., Yao, J.: Segmentation of motion textures using mixed-state Markov random fields. In: Proc. of SPIE, vol. 6315, p. 63150J. SPIE, Bellingham (2006) Google Scholar
  15. 15.
    Crivelli, T., Cernuschi-Frias, B., Bouthemy, P., Yao, J.F.: Mixed-state Markov random fields for motion texture modeling and segmentation. In: Proc. IEEE International Conference on Image Processing ICIP’06, Atlanta, USA, pp. 1857–1860 (2006) Google Scholar
  16. 16.
    Crivelli, T., Piriou, G., Bouthemy, P., Cernuschi-Frías, B., Yao, J.F.: Simultaneous motion detection and background reconstruction with a mixed-state conditional Markov random field. In: ECCV ’08: Proc. of the 10th European Conference on Computer Vision, Marseille, France, pp. 113–126 (2008) Google Scholar
  17. 17.
    Doretto, G., Chiuso, A., Wu, Y., Soatto, S.: Dynamic textures. Int. J. Comput. Vis. 51(2), 91–109 (2003) MATHCrossRefGoogle Scholar
  18. 18.
    Fablet, R., Bouthemy, P.: Motion recognition using non-parametric image motion models estimated from temporal and multiscale co-ocurrence statistics. IEEE Trans. Pattern Anal. Mach. Intell. 25(12), 1619–1624 (2003) CrossRefGoogle Scholar
  19. 19.
    Fablet, R., Bouthemy, P., Perez, P.: Non-parametric motion characterization using causal probabilistic models for video indexing and retrieval. IEEE Trans. Image Process. 11(4), 393–407 (2002) CrossRefGoogle Scholar
  20. 20.
    Fazekas, S., Chetverikov, D.: Normal versus complete flow in dynamic texture recognition: a comparative study. In: Texture 2005: 4th International Workshop on Texture Analysis and Synthesis, ICCV’05, Beijing, pp. 37–42 (2005) Google Scholar
  21. 21.
    Fazekas, S., Amiaz, T., Chetverikov, D., Kiryati, N.: Dynamic texture detection based on motion analysis. Int. J. Comput. Vis. 82(1), 48–63 (2009) CrossRefGoogle Scholar
  22. 22.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984) MATHCrossRefGoogle Scholar
  23. 23.
    Guyon, X.: Random Fields on a Network: Modeling, Statistics and Applications. Springer, New York (1995) MATHGoogle Scholar
  24. 24.
    Hardouin, C., Yao, J.F.: Spatial modelling for mixed-state observations. Electron. J. Stat. 2, 213–233 (2008) MathSciNetMATHCrossRefGoogle Scholar
  25. 25.
    Horn, B., Schunck, B.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981) CrossRefGoogle Scholar
  26. 26.
    Hsieh, J., Yu, S., Chen, Y.: Motion-based video retrieval by trajectory matching. IEEE Trans. Circuits Syst. Video Technol. 16(3), 396–409 (2006) CrossRefGoogle Scholar
  27. 27.
    Lu, Z., Xie, W., Pei, J., Huang, J.: Dynamic texture recognition by spatio-temporal multiresolution histograms. In: IEEE Workshop on Motion and Video Computing, WACV/MOTION, pp. 241–246 (2005) Google Scholar
  28. 28.
    Nelson, R.C., Polana, R.: Qualitative recognition of motion using temporal texture. CVGIP, Image Underst. 56(1), 78–89 (1992). doi: 10.1016/1049-9660(92)90087-J MATHCrossRefGoogle Scholar
  29. 29.
    Pérez, P., Hue, C., Vermaak, J., Gangnet, M.: Color-based probabilistic tracking. In: ECCV ’02: Proc. of the 7th European Conference on Computer Vision-Part I, pp. 661–675. Springer, London (2002) Google Scholar
  30. 30.
    Peteri, R., Chetverikov, D.: Dynamic texture recognition using normal flow and texture regularity. In: Proc. of IbPRIA, Estoril, pp. 223–230 (2005) Google Scholar
  31. 31.
    Peteri, R., Huiskes, M., Fazekas, S.: Dyntex: a comprehensive database of dynamic textures.
  32. 32.
    Piriou, G., Bouthemy, P., Yao, J.F.: Recognition of dynamic video contents with global probabilistic models of visual motion. IEEE Trans. Image Process. 15(11), 3417–3430 (2006) CrossRefGoogle Scholar
  33. 33.
    Rahman, A., Murshed, M.: Real-time temporal texture characterisation using block based motion co-occurrence statistics. In: Proc. of the 11th IEEE International Conference on Image Processing, ICIP’04, pp. 1593–1596 (2004) Google Scholar
  34. 34.
    Saisan, P., Doretto, G., Wu, Y., Soatto, S.: Dynamic texture recognition. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR’01, Hawaii, pp. 58–63 (2001) Google Scholar
  35. 35.
    Salzenstein, F., Collet, C.: Fuzzy Markov random fields versus chains for multispectral image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1753–1767 (2006) CrossRefGoogle Scholar
  36. 36.
    Salzenstein, F., Pieczynski, W.: Parameter estimation in hidden fuzzy Markov random fields and image segmentation. Graph. Models Image Process. 59(4), 205–220 (1997) CrossRefGoogle Scholar
  37. 37.
    Vidal, R., Ravichandran, A.: Optical flow estimation and segmentation of multiple moving dynamic textures. In: Proc. of CVPR’05, San Diego, vol. 2, pp. 516–521 (2005) Google Scholar
  38. 38.
    Wheeden, R., Zygmund, A.: Measure and Integral: An Introduction to Real Analysis. Dekker, New York (1977) MATHGoogle Scholar
  39. 39.
    Wu, J., Chung, A.C.S.: A segmentation model using compound Markov random fields based on a boundary model. IEEE Trans. Image Process. 16(1), 241–252 (2007).  10.1109/TIP.2006.884933 MathSciNetCrossRefGoogle Scholar
  40. 40.
    Yuan, L., Wen, F., Liu, C., Shum, H.: Synthesizing dynamic textures with closed-loop linear dynamic systems. In: Proc. of the 8th European Conference on Computer Vision, ECCV’04, Prague. Lecture Notes in Computer Science, vol. 3022, pp. 603–616. Springer, Berlin (2004) Google Scholar
  41. 41.
    Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–927 (2007) CrossRefGoogle Scholar
  42. 42.
    Zhu, S., Ma, K.K.: A new diamond search algorithm for fast block-matching motion estimation. IEEE Trans. Image Process. 9(2), 287–290 (2000) MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Tomás Crivelli
    • 1
  • Patrick Bouthemy
    • 2
  • Bruno Cernuschi Frías
    • 3
  • Jian-feng Yao
    • 4
  1. 1.University of Buenos AiresBuenos AiresArgentina
  2. 2.INRIARennesFrance
  3. 3.CONICETBuenos AiresArgentina
  4. 4.Unviersity of Rennes 1RennesFrance

Personalised recommendations