LBP-TOP: A Tensor Unfolding Revisit

  • Xiaopeng Hong
  • Yingyue Xu
  • Guoying ZhaoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10116)


Local Binary Pattern histograms from Three Orthogonal Planes (LBP-TOP) has shown its promising performance on facial expression recognition as well as human activity analysis, as it extracts features from spatial-temporal information. Originally, as the calculation of LBP-TOP has to traverse all the pixels in the three dimensional space to compute the LBP operation along XY, YT and XT planes respectively, the frequent use of loops in implementation shapely increases the computational costs. In this work, we aim to fasten the computational efficiency of LBP-TOP on spatial-temporal information and introduce the concept of tensor unfolding to accelerate the implementation process from three-dimensional space to two-dimensional space. The spatial-temporal information is interpreted as a 3-order tensor, and we use tensor unfolding method to compute three concatenated big matrices in two-dimensional space. LBP operation is then performed on the three unfolded matrices. As the demand for loops in implementation is largely down, the computational cost is substantially reduced. We compared the computational time of the original LBP-TOP implementation to that of our fast LBP-TOP implementation on both synthetic and real data, the results show that the fast LBP-TOP implementation is much more time-saving than the original one. The implementation code of the proposed fast LBP-TOP is now publicly available (The implementation code of the proposed fast LBP-TOP can be downloaded at



This work is sponsored by the Academy of Finland, Infotech Oulu, the post-doc fellow position of Infotech Oulu, and Tekes Fidipro Program. Moreover, Xiaopeng Hong is partly supported by the Natural Science Foundation of China under the contract No. 61572205. Also, we gratefully acknowledge the support of NVIDIA Corporation with the donation of the Tesla K40 GPU used for this research.


  1. 1.
    Wolf, L.: Face recognition, geometric vs. appearance-based. In: Encyclopedia of Biometrics, pp. 495–500 (2015)Google Scholar
  2. 2.
    Gross, R., Matthews, I., Baker, S.: Appearance-based face recognition and light-fields. IEEE Trans. Pattern Anal. Mach. Intell. 26, 449–465 (2004)CrossRefGoogle Scholar
  3. 3.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24, 971–987 (2002)CrossRefzbMATHGoogle Scholar
  4. 4.
    Ojala, T., Pietikäinen, M., Mäenpää, T.: Gray scale and rotation invariant texture classification with local binary patterns. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 404–420. Springer, Heidelberg (2000). doi: 10.1007/3-540-45054-8_27 CrossRefGoogle Scholar
  5. 5.
    Huang, X., Wang, S.J., Zhao, G., Piteikainen, M.: Facial micro-expression recognition using spatiotemporal local binary pattern with integral projection. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1–9 (2015)Google Scholar
  6. 6.
    Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29, 915–928 (2007)CrossRefGoogle Scholar
  7. 7.
    de Freitas Pereira, T., Anjos, A., De Martino, J.M., Marcel, S.: \({\mathit{LBP - TOP}}\) based countermeasure against face spoofing attacks. In: Park, J.-I., Kim, J. (eds.) ACCV 2012. LNCS, vol. 7728, pp. 121–132. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-37410-4_11 CrossRefGoogle Scholar
  8. 8.
    Kellokumpu, V., Zhao, G., Pietikäinen, M.: Human activity recognition using a dynamic texture based method. In: BMVC, vol. 1, p. 2 (2008)Google Scholar
  9. 9.
    Wang, Y., Yu, H., Stevens, B., Liu, H.: Dynamic facial expression recognition using local patch and lbp-top. In: 2015 8th International Conference on Human System Interaction (HSI), pp. 362–367. IEEE (2015)Google Scholar
  10. 10.
    Chen, Y., Guo, X., Klein, D.: Orthogonal combination of local binary patterns for dynamic texture recognition. In: Ninth International Symposium on Multispectral Image Processing and Pattern Recognition (MIPPR2015), 98130R. International Society for Optics and Photonics(2015)Google Scholar
  11. 11.
    Qi, X., Li, C.G., Zhao, G., Hong, X., Pietikäinen, M.: Dynamic texture and scene classification by transferring deep image features. Neurocomputing 171, 1230–1241 (2016)CrossRefGoogle Scholar
  12. 12.
    Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recogn. 29, 51–59 (1996)CrossRefGoogle Scholar
  13. 13.
    Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28, 2037–2041 (2006)CrossRefzbMATHGoogle Scholar
  14. 14.
    Ahonen, T., Hadid, A., Pietikäinen, M.: Face recognition with local binary patterns. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3021, pp. 469–481. Springer, Heidelberg (2004). doi: 10.1007/978-3-540-24670-1_36 CrossRefGoogle Scholar
  15. 15.
    Murala, S., Wu, Q.J.: Local mesh patterns versus local binary patterns: biomedical image indexing and retrieval. IEEE J. Biomed. Health Inform. 18, 929–938 (2014)CrossRefGoogle Scholar
  16. 16.
    Mäenpää, T.: The local binary pattern approach to texture analysis: extensions and applications. Oulun yliopisto (2003)Google Scholar
  17. 17.
    Kolecki, J.C.: An introduction to tensors for students of physics and engineering (2002)Google Scholar
  18. 18.
    Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51, 455–500 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Wang, S., Yan, W.J., Li, X., Zhao, G., Fu, X.: Micro-expression recognition using dynamic textures on tensor independent color space. In: ICPR, pp. 4678–4683. Citeseer (2014)Google Scholar
  20. 20.
    Wang, S.J., Yan, W.J., Li, X., Zhao, G., Zhou, C.G., Fu, X., Yang, M., Tao, J.: Micro-expression recognition using color spaces. IEEE Trans. Image Process. 24, 6034–6047 (2015)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Kim, T.K., Wong, S.F., Cipolla, R.: Tensor canonical correlation analysis for action classification. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)Google Scholar
  22. 22.
    Meyer, C.D.: Matrix Analysis and Applied Linear Algebra, vol. 2. SIAM, Philadelphia (2000)CrossRefGoogle Scholar
  23. 23.
    Kuang, L., Hao, F., Yang, L.T., Lin, M., Luo, C., Min, G.: A tensor-based approach for big data representation and dimensionality reduction. IEEE Trans. Emerg. Topics Comput. 2, 280–291 (2014)CrossRefGoogle Scholar
  24. 24.
    Manolopoulos, Y., Symeonidis, P., Nanopoulos, A.: Tag recommendations based on tensor dimensionality reduction (2008)Google Scholar
  25. 25.
    Yan, W.J., Li, X., Wang, S.J., Zhao, G., Liu, Y.J., Chen, Y.H., Fu, X.: Casme ii: an improved spontaneous micro-expression database and the baseline evaluation. PloS One 9, e86041 (2014)CrossRefGoogle Scholar
  26. 26.
    Li, X., Pfister, T., Huang, X., Zhao, G., Pietikäinen, M.: A spontaneous micro-expression database: Inducement, collection and baseline. In: 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–6. IEEE (2013)Google Scholar
  27. 27.
    Tian, Y.I., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23, 97–115 (2001)CrossRefGoogle Scholar
  28. 28.
    Almaev, T.R., Valstar, M.F.: Local gabor binary patterns from three orthogonal planes for automatic facial expression recognition. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII), pp. 356–361. IEEE (2013)Google Scholar
  29. 29.
    Sun, B., Li, L., Zhou, G., He, J.: Facial expression recognition in the wild based on multimodal texture features. J. Electron. Imaging 25, 061407 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Center for Machine Vision and Signal AnalysisUniversity of OuluOuluFinland

Personalised recommendations