Signal, Image and Video Processing

, Volume 13, Issue 3, pp 575–582 | Cite as

Sparse binarised statistical dynamic features for spatio-temporal texture analysis

  • Shervin Rahimzadeh ArashlooEmail author
Original Paper


The paper presents a new spatio-temporal learning-based descriptor called binarised statistical dynamic features (BSDF) for representation and classification of dynamic texture. The BSDF descriptor operates by applying three-dimensional spatio-temporal filters on local voxels of an image sequence where the filters are learned via an independent component analysis, maximising independence over spatial and temporal domains concurrently. The BSDF representation is formed by binarising filter responses which are then converted into codewords and summarised using histograms. A robust representation of the BSDF descriptor is finally obtained via a sparse representation approach yielding very discriminative features for classification. The effects of different hyper-parameters on performance including the number of filters, the number of scales, temporal depth, number of samples drawn are also investigated. The proposed approach is evaluated on the most commonly used dynamic texture databases and shown to perform very well compared to the existing methods.


Dynamic texture Spatio-temporal filtering Independent component analysis Sparse representation 


  1. 1.
  2. 2.
    Ali, W., Georgsson, F., Hellstrom, T.: Visual tree detection for autonomous navigation in forest environment. In: Intelligent Vehicles Symposium, 2008 IEEE, pp. 560–565 (2008).
  3. 3.
    Arashloo, S.R., Amirani, M.C., Noroozi, A.: Dynamic texture representation using a deep multi-scale convolutional network. J. Vis. Commun. Image Represent. 43, 89–97 (2017). CrossRefGoogle Scholar
  4. 4.
    Arashloo, S.R., Kittler, J.: Hierarchical image matching for pose-invariant face recognition. In: Cavallaro, A., Prince, S., Alexander D. (eds.) BMVC. British Machine Vision Association, London, UK (2009)Google Scholar
  5. 5.
    Arashloo, S.R., Kittler, J.: Dynamic texture recognition using multiscale binarized statistical image features. IEEE Trans. Multimed. 16(8), 2099–2109 (2014). CrossRefGoogle Scholar
  6. 6.
    Baktashmotlagh, M., Harandi, M., Lovell, B.C., Salzmann, M.: Discriminative non-linear stationary subspace analysis for video classification. IEEE Trans. Pattern Anal. Mach. Intell. 36(12), 2353–2366 (2014). CrossRefGoogle Scholar
  7. 7.
    Beham, M.P., Roomi, S.M.M.: Anti-spoofing enabled face recognition based on aggregated local weighted gradient orientation. Signal Image Video Process. 12(3), 531–538 (2018). CrossRefGoogle Scholar
  8. 8.
    Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013). CrossRefGoogle Scholar
  9. 9.
    Cannons, K.J., Gryn, J.M., Wildes, R.P.: Visual Tracking Using a Pixelwise Spatiotemporal Oriented Energy Representation, pp. 511–524. Springer, Berlin (2010)Google Scholar
  10. 10.
    Chan, A.B., Vasconcelos, N.: Probabilistic kernels for the classification of auto-regressive visual processes. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 846–851 (2005).
  11. 11.
    Chan, K.L.: Saliency detection in video sequences using perceivable change encoded local pattern. Signal Image Video Process. 12(5), 975–982 (2018). CrossRefGoogle Scholar
  12. 12.
    Chan, T.H., Jia, K., Gao, S., Lu, J., Zeng, Z., Ma, Y.: Pcanet: a simple deep learning baseline for image classification? IEEE Trans. Image Process. 24(12), 5017–5032 (2015). MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Chen, J., Zhao, G., Salo, M., Rahtu, E., Pietikainen, M.: Automatic dynamic texture segmentation using local descriptors and optical flow. IEEE Trans. Image Process. 22(1), 326–339 (2013). MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Culibrk, D., Sebe, N.: Temporal dropout of changes approach to convolutional learning of spatio-temporal features. In: K.A. Hua, Y. Rui, R. Steinmetz, A. Hanjalic, A. Natsev, W. Zhu (eds.) ACM Multimedia, pp. 1201–1204. ACM (2014)Google Scholar
  15. 15.
    Derpanis, K.G., Wildes, R.P.: Classification of traffic video based on a spatiotemporal orientation analysis. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV), pp. 606–613 (2011).
  16. 16.
    Derpanis, K.G.P., Wildes, R.: Spacetime texture representation and recognition based on a spatiotemporal orientation analysis. IEEE Trans. Pattern Anal. Mach. Intell. 34(6), 1193–1205 (2012). CrossRefGoogle Scholar
  17. 17.
    Derrac, J., Garca, S., Molina, D., Herrera, F.: A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol. Comput. 1(1), 3–18 (2011). CrossRefGoogle Scholar
  18. 18.
    Dollar, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 65–72 (2005).
  19. 19.
    Donoho, D.L., Tsaig, Y.: Fast solution of l1-norm minimization problems when the solution may be sparse. IEEE Trans. Inf. Theory 54(11), 4789–4812 (2008). CrossRefzbMATHGoogle Scholar
  20. 20.
    Doretto, G., Chiuso, A., Wu, Y.N., Soatto, S.: Dynamic textures. Int. J. Comput. Vis.: IJCV 51(2), 91–109 (2003)CrossRefzbMATHGoogle Scholar
  21. 21.
    Dubois, S., Pteri, R., Mnard, M.: Characterization and recognition of dynamic textures based on the 2d+t curvelet transform. Signal Image Video Process. 9(4), 819–830 (2015). CrossRefGoogle Scholar
  22. 22.
    Fitzgibbon, A.W.: Stochastic rigidity: image registration for nowhere-static scenes. In: Proceedings. Eighth IEEE International Conference on Computer Vision, 2001. ICCV 2001, vol. 1, pp. 662–669 (2001).
  23. 23.
    Ghanem, B., Ahuja, N.: Phase based modelling of dynamic textures. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8 (2007).
  24. 24.
    Ghanem, B., Ahuja, N.: Extracting a fluid dynamic texture and the background from video. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008, pp. 1–8 (2008).
  25. 25.
    Haas, M., Rijsdam, J., Thomee, B., Lew, M.S.: Relevance feedback: perceptual learning and retrieval in bio-computing, photos, and video. In: Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval, MIR ’04, pp. 151–156. ACM, New York, NY, USA (2004).
  26. 26.
    van Hateren, J.H., Ruderman, D.L.: Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex. Proc. R. Soc. Biol. Sci. 265(1412), 2315–2320 (1998). CrossRefGoogle Scholar
  27. 27.
    Hyvarinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw. 10(3), 626–634 (1999). CrossRefGoogle Scholar
  28. 28.
    Hyvrinen, A., Hurri, J., Hoyer, P.O.: Natural Image Statistics: A Probabilistic Approach to Early Computational Vision, 1st edn. Springer, Berlin (2009)CrossRefzbMATHGoogle Scholar
  29. 29.
    Ji, H., Yang, X., Ling, H., Xu, Y.: Wavelet domain multifractal analysis for static and dynamic texture classification. IEEE Trans. Image Process. 22(1), 286–299 (2013). MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Junejo, I.N., Bhutta, A.A., Foroosh, H.: Single-class svm for dynamic scene modeling. Signal Image Video Process. 7(1), 45–52 (2013). CrossRefGoogle Scholar
  31. 31.
    Kannala, J., Rahtu, E.: Bsif: Binarized statistical image features. In: 2012 21st International Conference on Pattern Recognition (ICPR), pp. 1363–1366 (2012)Google Scholar
  32. 32.
    Kung, T.J., Richards, W.: Inferring “water” from images. In: Richards, W. (ed.) Natural Computation, Chap. 16, pp. 224–233. M.I.T. Press, Cambridge, MA (1988)Google Scholar
  33. 33.
    Mumtaz, A., Coviello, E., Lanckriet, G.R.G., Chan, A.B.: Clustering dynamic textures with the hierarchical em algorithm for modeling video. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1606–1621 (2013). CrossRefGoogle Scholar
  34. 34.
    Nanni, L., Brahnam, S., Lumini, A.: Local ternary patterns from three orthogonal planes for human action classification. Expert Syst. Appl. 38(5), 5125–5128 (2011). CrossRefGoogle Scholar
  35. 35.
    Osborne, M., Presnell, B., Turlach, B.: A new approach to variable selection in least squares problems. IMA J. Numer. Anal. 20(3), 389 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Päivärinta, J., Rahtu, E., Heikkilä, J.: Volume Local Phase Quantization for Blur-Insensitive Dynamic Texture Classification, pp. 360–369. Springer, Berlin (2011)Google Scholar
  37. 37.
    Péteri, R., Fazekas, S., Huiskes, M.J.: DynTex : a comprehensive database of dynamic textures. Pattern Recogn. Lett.
  38. 38.
    Qi, X., Li, C.G., Zhao, G., Hong, X., Pietikainen, M.: Dynamic texture and scene classification by transferring deep image features. Neurocomputing 171, 1230–1241 (2016). CrossRefGoogle Scholar
  39. 39.
    Qiao, Y., Weng, L.: Hidden markov model based dynamic texture classification. IEEE Signal Process. Lett. 22(4), 509–512 (2015). CrossRefGoogle Scholar
  40. 40.
    Quan, Y., Huang, Y., Ji, H.: Dynamic texture recognition via orthogonal tensor dictionary learning. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 73–81 (2015).
  41. 41.
    Ravichandran, A., Chaudhry, R., Vidal, R.: View-invariant dynamic texture recognition using a bag of dynamical systems. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 1651–1657 (2009).
  42. 42.
    Ravichandran, A., Chaudhry, R., Vidal, R.: Categorizing dynamic textures using a bag of dynamical systems. IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 342–353 (2013). CrossRefGoogle Scholar
  43. 43.
    Rivera, A.R., Chae, O.: Spatiotemporal directional number transitional graph for dynamic texture recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(10), 2146–2152 (2015). CrossRefGoogle Scholar
  44. 44.
    Saisan, P., Doretto, G., Wu, Y.N., Soatto, S.: Dynamic texture recognition. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. CVPR 2001, vol. 2, pp. II-58–II-63 (2001).
  45. 45.
    Thriault, C., Thome, N., Cord, M.: Dynamic scene classification: learning motion descriptors with slow features analysis. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2603–2610 (2013).
  46. 46.
    Wang, Y., Chun Zhu, S.: Modeling textured motion: particle, wave and sketch. In: IEEE International Conference on Computer Vision, ICCV’03, pp. 213–220 (2003)Google Scholar
  47. 47.
    Wildes, R.P., Bergen, J.R.: Qualitative Spatiotemporal Analysis Using an Oriented Energy Representation, pp. 768–784. Springer, Berlin (2000)Google Scholar
  48. 48.
    Woolfe, F., Fitzgibbon, A.W.: Shift-invariant dynamic texture recognition. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV (2), Lecture Notes in Computer Science, vol. 3952, pp. 549–562. Springer, Berlin (2006)Google Scholar
  49. 49.
    Xie, J., Fang, Y.: Dynamic texture recognition with video set based collaborative representation. Image Vis. Comput. 55(Part 2), 86–92 (2016)CrossRefGoogle Scholar
  50. 50.
    Zhang, Z., Xu, Y., Yang, J., Li, X., Zhang, D.: A survey of sparse representation: algorithms and applications. IEEE Access 3, 490–530 (2015). CrossRefGoogle Scholar
  51. 51.
    Zhao, G., Barnard, M., Pietikainen, M.: Lipreading with local spatiotemporal descriptors. IEEE Trans. Multimed. 11(7), 1254–1265 (2009). CrossRefGoogle Scholar
  52. 52.
    Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007). CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer Engineering, Faculty of EngineeringBilkent UniversityAnkaraTurkey

Personalised recommendations