Advertisement

Signal, Image and Video Processing

, Volume 13, Issue 3, pp 575–582 | Cite as

Sparse binarised statistical dynamic features for spatio-temporal texture analysis

  • Shervin Rahimzadeh ArashlooEmail author
Original Paper
  • 97 Downloads

Abstract

The paper presents a new spatio-temporal learning-based descriptor called binarised statistical dynamic features (BSDF) for representation and classification of dynamic texture. The BSDF descriptor operates by applying three-dimensional spatio-temporal filters on local voxels of an image sequence where the filters are learned via an independent component analysis, maximising independence over spatial and temporal domains concurrently. The BSDF representation is formed by binarising filter responses which are then converted into codewords and summarised using histograms. A robust representation of the BSDF descriptor is finally obtained via a sparse representation approach yielding very discriminative features for classification. The effects of different hyper-parameters on performance including the number of filters, the number of scales, temporal depth, number of samples drawn are also investigated. The proposed approach is evaluated on the most commonly used dynamic texture databases and shown to perform very well compared to the existing methods.

Keywords

Dynamic texture Spatio-temporal filtering Independent component analysis Sparse representation 

References

  1. 1.
  2. 2.
    Ali, W., Georgsson, F., Hellstrom, T.: Visual tree detection for autonomous navigation in forest environment. In: Intelligent Vehicles Symposium, 2008 IEEE, pp. 560–565 (2008).  https://doi.org/10.1109/IVS.2008.4621315
  3. 3.
    Arashloo, S.R., Amirani, M.C., Noroozi, A.: Dynamic texture representation using a deep multi-scale convolutional network. J. Vis. Commun. Image Represent. 43, 89–97 (2017).  https://doi.org/10.1016/j.jvcir.2016.12.015 CrossRefGoogle Scholar
  4. 4.
    Arashloo, S.R., Kittler, J.: Hierarchical image matching for pose-invariant face recognition. In: Cavallaro, A., Prince, S., Alexander D. (eds.) BMVC. British Machine Vision Association, London, UK (2009)Google Scholar
  5. 5.
    Arashloo, S.R., Kittler, J.: Dynamic texture recognition using multiscale binarized statistical image features. IEEE Trans. Multimed. 16(8), 2099–2109 (2014).  https://doi.org/10.1109/TMM.2014.2362855 CrossRefGoogle Scholar
  6. 6.
    Baktashmotlagh, M., Harandi, M., Lovell, B.C., Salzmann, M.: Discriminative non-linear stationary subspace analysis for video classification. IEEE Trans. Pattern Anal. Mach. Intell. 36(12), 2353–2366 (2014).  https://doi.org/10.1109/TPAMI.2014.2339851 CrossRefGoogle Scholar
  7. 7.
    Beham, M.P., Roomi, S.M.M.: Anti-spoofing enabled face recognition based on aggregated local weighted gradient orientation. Signal Image Video Process. 12(3), 531–538 (2018).  https://doi.org/10.1007/s11760-017-1189-1 CrossRefGoogle Scholar
  8. 8.
    Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013).  https://doi.org/10.1109/TPAMI.2013.50 CrossRefGoogle Scholar
  9. 9.
    Cannons, K.J., Gryn, J.M., Wildes, R.P.: Visual Tracking Using a Pixelwise Spatiotemporal Oriented Energy Representation, pp. 511–524. Springer, Berlin (2010)Google Scholar
  10. 10.
    Chan, A.B., Vasconcelos, N.: Probabilistic kernels for the classification of auto-regressive visual processes. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 846–851 (2005).  https://doi.org/10.1109/CVPR.2005.279
  11. 11.
    Chan, K.L.: Saliency detection in video sequences using perceivable change encoded local pattern. Signal Image Video Process. 12(5), 975–982 (2018).  https://doi.org/10.1007/s11760-018-1242-8 CrossRefGoogle Scholar
  12. 12.
    Chan, T.H., Jia, K., Gao, S., Lu, J., Zeng, Z., Ma, Y.: Pcanet: a simple deep learning baseline for image classification? IEEE Trans. Image Process. 24(12), 5017–5032 (2015).  https://doi.org/10.1109/TIP.2015.2475625 MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Chen, J., Zhao, G., Salo, M., Rahtu, E., Pietikainen, M.: Automatic dynamic texture segmentation using local descriptors and optical flow. IEEE Trans. Image Process. 22(1), 326–339 (2013).  https://doi.org/10.1109/TIP.2012.2210234 MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Culibrk, D., Sebe, N.: Temporal dropout of changes approach to convolutional learning of spatio-temporal features. In: K.A. Hua, Y. Rui, R. Steinmetz, A. Hanjalic, A. Natsev, W. Zhu (eds.) ACM Multimedia, pp. 1201–1204. ACM (2014)Google Scholar
  15. 15.
    Derpanis, K.G., Wildes, R.P.: Classification of traffic video based on a spatiotemporal orientation analysis. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV), pp. 606–613 (2011).  https://doi.org/10.1109/WACV.2011.5711560
  16. 16.
    Derpanis, K.G.P., Wildes, R.: Spacetime texture representation and recognition based on a spatiotemporal orientation analysis. IEEE Trans. Pattern Anal. Mach. Intell. 34(6), 1193–1205 (2012).  https://doi.org/10.1109/TPAMI.2011.221 CrossRefGoogle Scholar
  17. 17.
    Derrac, J., Garca, S., Molina, D., Herrera, F.: A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol. Comput. 1(1), 3–18 (2011).  https://doi.org/10.1016/j.swevo.2011.02.002 CrossRefGoogle Scholar
  18. 18.
    Dollar, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 65–72 (2005).  https://doi.org/10.1109/VSPETS.2005.1570899
  19. 19.
    Donoho, D.L., Tsaig, Y.: Fast solution of l1-norm minimization problems when the solution may be sparse. IEEE Trans. Inf. Theory 54(11), 4789–4812 (2008).  https://doi.org/10.1109/TIT.2008.929958 CrossRefzbMATHGoogle Scholar
  20. 20.
    Doretto, G., Chiuso, A., Wu, Y.N., Soatto, S.: Dynamic textures. Int. J. Comput. Vis.: IJCV 51(2), 91–109 (2003)CrossRefzbMATHGoogle Scholar
  21. 21.
    Dubois, S., Pteri, R., Mnard, M.: Characterization and recognition of dynamic textures based on the 2d+t curvelet transform. Signal Image Video Process. 9(4), 819–830 (2015).  https://doi.org/10.1007/s11760-013-0532-4 CrossRefGoogle Scholar
  22. 22.
    Fitzgibbon, A.W.: Stochastic rigidity: image registration for nowhere-static scenes. In: Proceedings. Eighth IEEE International Conference on Computer Vision, 2001. ICCV 2001, vol. 1, pp. 662–669 (2001).  https://doi.org/10.1109/ICCV.2001.937584
  23. 23.
    Ghanem, B., Ahuja, N.: Phase based modelling of dynamic textures. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8 (2007).  https://doi.org/10.1109/ICCV.2007.4409094
  24. 24.
    Ghanem, B., Ahuja, N.: Extracting a fluid dynamic texture and the background from video. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008, pp. 1–8 (2008).  https://doi.org/10.1109/CVPR.2008.4587547
  25. 25.
    Haas, M., Rijsdam, J., Thomee, B., Lew, M.S.: Relevance feedback: perceptual learning and retrieval in bio-computing, photos, and video. In: Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval, MIR ’04, pp. 151–156. ACM, New York, NY, USA (2004).  https://doi.org/10.1145/1026711.1026737
  26. 26.
    van Hateren, J.H., Ruderman, D.L.: Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex. Proc. R. Soc. Biol. Sci. 265(1412), 2315–2320 (1998).  https://doi.org/10.1098/rspb.1998.0577 CrossRefGoogle Scholar
  27. 27.
    Hyvarinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw. 10(3), 626–634 (1999).  https://doi.org/10.1109/72.761722 CrossRefGoogle Scholar
  28. 28.
    Hyvrinen, A., Hurri, J., Hoyer, P.O.: Natural Image Statistics: A Probabilistic Approach to Early Computational Vision, 1st edn. Springer, Berlin (2009)CrossRefzbMATHGoogle Scholar
  29. 29.
    Ji, H., Yang, X., Ling, H., Xu, Y.: Wavelet domain multifractal analysis for static and dynamic texture classification. IEEE Trans. Image Process. 22(1), 286–299 (2013).  https://doi.org/10.1109/TIP.2012.2214040 MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Junejo, I.N., Bhutta, A.A., Foroosh, H.: Single-class svm for dynamic scene modeling. Signal Image Video Process. 7(1), 45–52 (2013).  https://doi.org/10.1007/s11760-011-0230-z CrossRefGoogle Scholar
  31. 31.
    Kannala, J., Rahtu, E.: Bsif: Binarized statistical image features. In: 2012 21st International Conference on Pattern Recognition (ICPR), pp. 1363–1366 (2012)Google Scholar
  32. 32.
    Kung, T.J., Richards, W.: Inferring “water” from images. In: Richards, W. (ed.) Natural Computation, Chap. 16, pp. 224–233. M.I.T. Press, Cambridge, MA (1988)Google Scholar
  33. 33.
    Mumtaz, A., Coviello, E., Lanckriet, G.R.G., Chan, A.B.: Clustering dynamic textures with the hierarchical em algorithm for modeling video. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1606–1621 (2013).  https://doi.org/10.1109/TPAMI.2012.236 CrossRefGoogle Scholar
  34. 34.
    Nanni, L., Brahnam, S., Lumini, A.: Local ternary patterns from three orthogonal planes for human action classification. Expert Syst. Appl. 38(5), 5125–5128 (2011).  https://doi.org/10.1016/j.eswa.2010.09.137 CrossRefGoogle Scholar
  35. 35.
    Osborne, M., Presnell, B., Turlach, B.: A new approach to variable selection in least squares problems. IMA J. Numer. Anal. 20(3), 389 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Päivärinta, J., Rahtu, E., Heikkilä, J.: Volume Local Phase Quantization for Blur-Insensitive Dynamic Texture Classification, pp. 360–369. Springer, Berlin (2011)Google Scholar
  37. 37.
    Péteri, R., Fazekas, S., Huiskes, M.J.: DynTex : a comprehensive database of dynamic textures. Pattern Recogn. Lett.  https://doi.org/10.1016/j.patrec.2010.05.009
  38. 38.
    Qi, X., Li, C.G., Zhao, G., Hong, X., Pietikainen, M.: Dynamic texture and scene classification by transferring deep image features. Neurocomputing 171, 1230–1241 (2016).  https://doi.org/10.1016/j.neucom.2015.07.071 CrossRefGoogle Scholar
  39. 39.
    Qiao, Y., Weng, L.: Hidden markov model based dynamic texture classification. IEEE Signal Process. Lett. 22(4), 509–512 (2015).  https://doi.org/10.1109/LSP.2014.2362613 CrossRefGoogle Scholar
  40. 40.
    Quan, Y., Huang, Y., Ji, H.: Dynamic texture recognition via orthogonal tensor dictionary learning. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 73–81 (2015).  https://doi.org/10.1109/ICCV.2015.17
  41. 41.
    Ravichandran, A., Chaudhry, R., Vidal, R.: View-invariant dynamic texture recognition using a bag of dynamical systems. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 1651–1657 (2009).  https://doi.org/10.1109/CVPR.2009.5206847
  42. 42.
    Ravichandran, A., Chaudhry, R., Vidal, R.: Categorizing dynamic textures using a bag of dynamical systems. IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 342–353 (2013).  https://doi.org/10.1109/TPAMI.2012.83 CrossRefGoogle Scholar
  43. 43.
    Rivera, A.R., Chae, O.: Spatiotemporal directional number transitional graph for dynamic texture recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(10), 2146–2152 (2015).  https://doi.org/10.1109/TPAMI.2015.2392774 CrossRefGoogle Scholar
  44. 44.
    Saisan, P., Doretto, G., Wu, Y.N., Soatto, S.: Dynamic texture recognition. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. CVPR 2001, vol. 2, pp. II-58–II-63 (2001).  https://doi.org/10.1109/CVPR.2001.990925
  45. 45.
    Thriault, C., Thome, N., Cord, M.: Dynamic scene classification: learning motion descriptors with slow features analysis. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2603–2610 (2013).  https://doi.org/10.1109/CVPR.2013.336
  46. 46.
    Wang, Y., Chun Zhu, S.: Modeling textured motion: particle, wave and sketch. In: IEEE International Conference on Computer Vision, ICCV’03, pp. 213–220 (2003)Google Scholar
  47. 47.
    Wildes, R.P., Bergen, J.R.: Qualitative Spatiotemporal Analysis Using an Oriented Energy Representation, pp. 768–784. Springer, Berlin (2000)Google Scholar
  48. 48.
    Woolfe, F., Fitzgibbon, A.W.: Shift-invariant dynamic texture recognition. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV (2), Lecture Notes in Computer Science, vol. 3952, pp. 549–562. Springer, Berlin (2006)Google Scholar
  49. 49.
    Xie, J., Fang, Y.: Dynamic texture recognition with video set based collaborative representation. Image Vis. Comput. 55(Part 2), 86–92 (2016)CrossRefGoogle Scholar
  50. 50.
    Zhang, Z., Xu, Y., Yang, J., Li, X., Zhang, D.: A survey of sparse representation: algorithms and applications. IEEE Access 3, 490–530 (2015).  https://doi.org/10.1109/ACCESS.2015.2430359 CrossRefGoogle Scholar
  51. 51.
    Zhao, G., Barnard, M., Pietikainen, M.: Lipreading with local spatiotemporal descriptors. IEEE Trans. Multimed. 11(7), 1254–1265 (2009).  https://doi.org/10.1109/TMM.2009.2030637 CrossRefGoogle Scholar
  52. 52.
    Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007).  https://doi.org/10.1109/TPAMI.2007.1110 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer Engineering, Faculty of EngineeringBilkent UniversityAnkaraTurkey

Personalised recommendations