A generic codebook based approach for gait recognition

  • Muhammad Hassan KhanEmail author
  • Muhammad Shahid Farid
  • Marcin Grzegorzek


Gait refers to the walking style of a person and it has emerged as an important biometric feature for person identification. The gait recognition algorithms proposed in literature exploit various types of information from the gait video sequence, such as, the skeletal data, human body shape, and silhouettes; and use these features to recognize the individuals. This paper presents the proposal of using a generic codebook in gait recognition. The idea is built upon a novel gait representation which exploits the spatiotemporal motion characteristics of the individual for identification. In particular, we propose to use a set of sample gait sequences to construct a generic codebook and use it to build a gait signature for person identification. To this end, we chose synthetic gait sequences of CMU MoCap gait database due to its diversity in walking styles. A set of spatiotemporal features are extracted from these sequences to build a generic codebook. The motion descriptors of real gait sequences are encoded using this generic codebook and Fisher vector encoding; the classification is performed using support vector machine. An extensive evaluation of this novel proposal is carried out using five benchmark gait databases: NLPR, CMU MoBo, TUM GAID, CASIA-B, and CASISA-C. In all experiments, the generic codebook is used in feature encoding. The performance of the proposed algorithm is also compared with the state-of-the-art gait recognition techniques and the results show that the idea of using a generic codebook in gait recognition is practical and effective.


Gait recognition Codebook Spatiotemporal features Fisher vector encoding Feature evaluation 



  1. 1.
    Anzai Y (2012) Pattern recognition and machine learning. Elsevier, AmsterdamzbMATHGoogle Scholar
  2. 2.
    Bashir K, Xiang T, Gong S (2008) Feature selection for gait recognition without subject cooperation. In: BMVC, pp 1–10Google Scholar
  3. 3.
    Bashir K, Xiang T, Gong S (2009) Gait recognition using gait entropy image. In: IET ICDP, pp 1–6Google Scholar
  4. 4.
    Bashir K, Xiang T, Gong S (2010) Gait recognition without subject cooperation. Pattern Recognit Lett 31(13):2052–2060Google Scholar
  5. 5.
    Bay H, Tuytelaars T, Van Gool L (2006) Surf: speeded up robust features. In: ECCV 2006, pp 404–417Google Scholar
  6. 6.
    BenAbdelkader C, Cutler RG, Davis LS (2004) Gait recognition using image self-similarity. EURASIP J Adv Signal Process 2004(4):1–14Google Scholar
  7. 7.
    Bouchrika I, Carter JN, Nixon M (2016) Towards automated visual surveillance using gait for identity recognition and tracking across multiple non-intersecting cameras. Multimed Tools Appl 75(2):1201–1221Google Scholar
  8. 8.
    Bouchrika I, Nixon M (2007) Model-based feature extraction for gait analysis and recognition. In: ICCV. Springer, pp 150–160Google Scholar
  9. 9.
    Castro FM, Marín-Jiménez MJ, Mata NG, Muñoz-Salinas R (2017) Fisher motion descriptor for multiview gait recognition. Int J Pattern Recognit Artif Intell 31(01):1756002. Google Scholar
  10. 10.
    Castro FM, Marín-jiménez MJ, Guil N, de la Blanca NP (2017) Automatic learning of gait signatures for people identification. In: International work-conference on artificial neural networks. Springer, pp 257–270Google Scholar
  11. 11.
    Chai Y, et al. (2006) A novel human gait recognition method by segmenting and extracting the region variance feature. In: Proc. Int. Conf. Pattern Recognit. (ICPR), vol 4, pp 425–428Google Scholar
  12. 12.
    Chen C, et al. (2009) Frame difference energy image for gait recognition with incomplete silhouettes. Pattern Recognit Lett 30(11):977–984Google Scholar
  13. 13.
    Chen S, Gao Y (2007) An invariant appearance model for gait recognition. In: Proc. IEEE Int. Conf. Multimed. and expo (ICME). IEEE, pp 1375–1378Google Scholar
  14. 14.
    Choudhury SD, Tjahjadi T (2012) Silhouette-based gait recognition using procrustes shape analysis and elliptic fourier descriptors. Pattern Recognit 45 (9):3414–3426Google Scholar
  15. 15.
    CMU motion capture database.
  16. 16.
    Cunado D, Nixon M, Carter JN (2003) Automatic extraction and description of human gait models for recognition purposes. Comput Vis Image Underst 90(1):1–41Google Scholar
  17. 17.
    Dadashi F, et al. (2009) Gait recognition using wavelet packet silhouette representation and transductive support vector machines. In: 2nd Int. congress on image and signal process, pp 1–5Google Scholar
  18. 18.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE CVPR, vol 1, pp 886–893.
  19. 19.
    Dalal N, Triggs B, Schmid C (2006) Human detection using oriented histograms of flow and appearance. In: ECCV, pp 428–441Google Scholar
  20. 20.
    DeCann B, Ross A (2010) Gait curves for human recognition, backpack detection, and silhouette correction in a nighttime environment. In: SPIE defense, security, and sensing. International society for optics and photonics, pp 76670q–76670qGoogle Scholar
  21. 21.
    Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the em algorithm. J R Stat Soc Ser B-Stat Methodol 1977:1–38MathSciNetzbMATHGoogle Scholar
  22. 22.
    Dupuis Y, Savatier X, Vasseur P (2013) Feature subset selection applied to model-free gait recognition. Image Vis Comput 31(8):580–591Google Scholar
  23. 23.
    Fan RE, et al. (2008) Liblinear: a library for large linear classification. J Mach Learn Res 9(Aug):1871–1874zbMATHGoogle Scholar
  24. 24.
    Geng X, Wang L, Li M, Wu Q, Smith-Miles K (2007) Distance-driven fusion of gait and face for human identification in video. In: Image and vision computing conference. Image and vision computing New ZealandGoogle Scholar
  25. 25.
    Goffredo M, Bouchrika I, Carter JN, Nixon M (2010) Performance analysis for automated gait extraction and recognition in multi-camera surveillance. Multimed. Tools Appl. 50(1):75–94Google Scholar
  26. 26.
    Goffredo M, Carter JN, Nixon M (2008) Front-view gait recognition. In: IEEE Int. Conf. Biometrics: theory, Appl. and Systems (BTAS). IEEE, pp 1–6Google Scholar
  27. 27.
    Gross R, Shi J (2001) The CMU motion of body (mobo) database. Gait Video SequencesGoogle Scholar
  28. 28.
    Hofmann M, Bachmann S, Rigoll G (2012) 2.5D gait biometrics using the depth gradient histogram energy image. In: IEEE BATS Conf., pp 399–403Google Scholar
  29. 29.
    Hu M, Wang Y, Zhang Z (2013) Cross-view gait recognition with short probe sequences: from view transformation model to view-independent stance-independent identity vector. Int J Pattern Recognit Artif Intell 27 (06):1350017. Google Scholar
  30. 30.
    Hu M, Wang Y, Zhang Z, Zhang D, Little JJ (2013) Incremental learning for video-based gait recognition with lbp flow. IEEE Trans Cybern 43(1):77–89Google Scholar
  31. 31.
    Kale A, Cuntoor N, Yegnanarayana B, Rajagopalan A, Chellappa R (2003) Gait analysis for human identification. In: Int. Conf. on audio-and video-based biometric person authentication. Springer, pp 706–714Google Scholar
  32. 32.
    Kale A, Sundaresan A, Rajagopalan A, Cuntoor NP, Roy-Chowdhury AK, Kruger V, Chellappa R (2004) Identification of humans using gait. IEEE Trans Image Process 13(9):1163–1173Google Scholar
  33. 33.
    Khan M, et al. (2016) Automatic recognition of movement patterns in the vojta-therapy using rgb-d data. In: Proc. Int. Conf. Image Process. (ICIP), pp 1235–1239Google Scholar
  34. 34.
    Khan MH (2018) Human activity analysis in visual surveillance and healthcare, vol 45. Logos Verlag Berlin GmbH, BerlinGoogle Scholar
  35. 35.
    Khan MH, Farid MS, Grzegorzek M (2017) Person identification using spatiotemporal motion characteristics. In: Proc. Int. Conf. Image Process. (ICIP). IEEE, pp 166–170Google Scholar
  36. 36.
    Khan MH, Farid MS, Grzegorzek M (2018) Using a generic model for codebook-based gait recognition algorithms. In: Int. workshop biometrics forensics (IWBF). IEEE, pp 1–7Google Scholar
  37. 37.
    Khan MH, Farid MS, Grzegorzek M (2019) Spatiotemporal features of human motion for gait recognition. Signal Image Video Process 13(2):369–377Google Scholar
  38. 38.
    Khan MH, Farid MS, Zahoor M, Grzegorzek M (2018) Cross-view gait recognition using non-linear view transformations of spatiotemporal features. In: Proc. Int. Conf. Image Process. (ICIP). IEEE, pp 773–777Google Scholar
  39. 39.
    Khan MH, Helsper J, Farid MS, Grzegorzek M (2018) A computer vision-based system for monitoring vojta therapy. J Med Informat 113:85–95Google Scholar
  40. 40.
    Khan MH, Li F, Farid MS, Grzegorzek M Kurzynski M, Wozniak M, Burduk R (eds) (2017) Gait recognition using motion trajectory analysis. Springer, ChamGoogle Scholar
  41. 41.
    Kusakunniran W (2014) Attribute-based learning for gait recognition using spatio-temporal interest points. Image Vis Comput 32(12):1117–1126Google Scholar
  42. 42.
    Kusakunniran W, Wu Q, Li H, Zhang J (2009) Automatic gait recognition using weighted binary pattern on video. In: IEEE Int. Conf. Adv. video signal based surveillance (AVSS). IEEE, pp 49–54Google Scholar
  43. 43.
    Kusakunniran W, Wu Q, Zhang J, Li H (2011) Pairwise shape configuration-based psa for gait recognition under small viewing angle change. In: IEEE Int. Conf. Adv. Video signal based surveillance (AVSS). IEEE, pp 17–22Google Scholar
  44. 44.
    Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: IEEE CVPR, pp 1–8, DOI
  45. 45.
    Lee H, Hong S, Kim E (2008) An efficient gait recognition based on a selective neural network ensemble. Int J Imaging Syst Technol 18(4):237–241Google Scholar
  46. 46.
    Lee L, Grimson WEL (2002) Gait analysis for recognition and classification. In: Proc. Int. Conf. Automatic face and gesture recognit. IEEE, pp 155–162Google Scholar
  47. 47.
    Liang J, Chen Y, Hu H, Zhao H (2006) Appearance-based gait recognition using independent component analysis. In: Int. Conf. on natural computation. Springer, pp 371–380Google Scholar
  48. 48.
    Loula F, Prasad S, Harber K, Shiffrar M (2005) Recognizing people from their movement. J Exp Psychol-Hum Percept 31(1):210Google Scholar
  49. 49.
    Lowe DG (1999) Object recognition from local scale-invariant features. In: EEE ICCV, vol 2, pp 1150–1157 vol.2.
  50. 50.
    Lu J, Zhang E, Jing C (2006) Gait recognition using wavelet descriptors and independent component analysis. In: Int. Symp. Neural networks. Springer, pp 232–237Google Scholar
  51. 51.
    Lun R, Zhao W (2015) A survey of applications and human motion recognition with microsoft kinect. Int J Pattern Recognit Artif Intell 29 (05):1555008. Google Scholar
  52. 52.
    Man J, Bhanu B (2006) Individual recognition using gait energy image. IEEE Trans Pattern Anal Mach Intell 28(2):316–322Google Scholar
  53. 53.
    Nixon M, et al. (2009) Model-based gait recognition. In: Enclycopedia of biometrics. Springer, pp 633–639.
  54. 54.
    Peng X, Wang L, Wang X, Qiao Y (2016) Bag of visual words and fusion methods for action recognition: comprehensive study and good practice. Comput Vis Image Underst 150:109–125Google Scholar
  55. 55.
    Perronnin F, Sánchez J, Mensink T (2010) Improving the fisher kernel for large-scale image classification. In: ECCV. Springer, pp 143–156Google Scholar
  56. 56.
    Rokanujjaman M, Islam MS, Hossain MA, Islam MR, Makihara Y, Yagi Y (2015) Effective part-based gait identification using frequency-domain gait entropy features. Multimed Tools Appl 74(9):3099–3120Google Scholar
  57. 57.
    Samangooei S, Nixon M (2010) Performing content-based retrieval of humans using gait biometrics. Multimed Tools Appl 49(1):195–212Google Scholar
  58. 58.
    Sánchez J, et al. (2013) Image classification with the fisher vector: theory and practice. Int J Comput Vis 105(3):222–245MathSciNetzbMATHGoogle Scholar
  59. 59.
    Shaikh SH, Saeed K, Chaki N (2014) Gait recognition using partial silhouette-based approach. In: Int. Conf. Signal process. and integrated netw. (SPIN). IEEE, pp 101–106Google Scholar
  60. 60.
    Sivapalan S, Chen D, Denman S, Sridharan S (2011) Fookes, c.: 3d ellipsoid fitting for multi-view gait recognition. In: IEEE Int. Conf. Adv. Video signal based surveillance (AVSS). IEEE, pp 355–360Google Scholar
  61. 61.
    Su H, Huang F (2006) Gait recognition using principal curves and neural networks. In: Int. Symp. Neural Networks. Springer, pp 238–243Google Scholar
  62. 62.
    Sun C, Nevatia R (2013) Large-scale web video event classification by use of fisher vectors. In: IEEE Int. Workshop Appl. Comput. Vis. (WACV). IEEE, pp 15–22Google Scholar
  63. 63.
    Tan D, Huang K, Yu S, Tan T (2007) Orthogonal diagonal projections for gait recognition. In: Proc. Int. Conf. Image process. (ICIP), vol 1. IEEE, pp i–337Google Scholar
  64. 64.
    Tan D, Huang K, Yu S, Tan T (2007) Recognizing night walkers based on one pseudoshape representation of gait. In: Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern recognit. (CVPR). IEEE, pp 1–8Google Scholar
  65. 65.
    Tan D, Huang K, Yu S, Tan T (2007) Uniprojective features for gait recognition. In: Int. Conf. Biometrics (ICB). Springer, pp 673–682Google Scholar
  66. 66.
    Tan D, Yu S, Huang K, Tan T (2007) Walker recognition without gait cycle estimation. In: Int. Conf. on biometrics, pp 222–231Google Scholar
  67. 67.
    Tan D, et al. (2006) Efficient night gait recognition based on template matching. In: Proc. Int. Conf. Pattern recognit. (ICPR), vol 3, pp 1000–1003Google Scholar
  68. 68.
    Veeraraghavan A, Chowdhury AR, Chellappa R (2004) Role of shape and kinematics in human movement analysis. In: Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR), vol 1. IEEE, pp i–730Google Scholar
  69. 69.
    Veeraraghavan A, Roy-Chowdhury AK, Chellappa R (2005) Matching shape sequences in video with applications in human movement analysis. IEEE Trans Pattern Anal Mach Intell 27(12):1896–1909Google Scholar
  70. 70.
    Wan M, Lai Z, Yang G, Yang Z, Zhang F, Zheng H (2017) Local graph embedding based on maximum margin criterion via fuzzy set. Fuzzy Sets Syst 318:120–131MathSciNetGoogle Scholar
  71. 71.
    Wan M, Li M, Yang G, Gai S, Jin Z (2014) Feature extraction using two-dimensional maximum embedding difference. Information Science 274:55–69Google Scholar
  72. 72.
    Wan M, Yang G, Gai S, Yang Z (2017) Two-dimensional discriminant locality preserving projections (2ddlpp) and its application to feature extraction via fuzzy set. Multimed Tools Appl 76(1):355–371Google Scholar
  73. 73.
    Wang C, et al. (2012) Human identification using temporal information preserving gait template. IEEE Trans Pattern Anal Mach Intell 34(11):2164–2176Google Scholar
  74. 74.
    Wang H, Schmid C (2013) Action recognition with improved trajectories. In: IEEE ICCV, pp 3551–3558Google Scholar
  75. 75.
    Wang L, Ning H, Tan T, Hu W (2004) Fusion of static and dynamic body biometrics for gait recognition. IEEE Trans Circuits Syst Video Technol 14(2):149–158Google Scholar
  76. 76.
    Wang L, Tan T, Hu W, Ning H (2003) Automatic gait recognition based on statistical shape analysis. IEEE Trans Image Process 12(9):1120–1131MathSciNetGoogle Scholar
  77. 77.
    Wang L, Tan T, Ning H, Hu W (2003) Silhouette analysis-based gait recognition for human identification. IEEE Trans Pattern Anal Mach Intell 25 (12):1505–1518Google Scholar
  78. 78.
    Whytock T, Belyaev A, Robertson N (2014) Dynamic distance-based shape features for gait recognition. J Math Imaging Vis 50(3):314–326zbMATHGoogle Scholar
  79. 79.
    Wu Q, Wang L, Geng X, Li M, He X (2007) Dynamic biometrics fusion at feature level for video based human recognition. In: Proc. of image and Vis. Computing New Zealand. Citeseer, pp 152–157Google Scholar
  80. 80.
    Yang Y, Tu D, Li G (2014) Gait recognition using flow histogram energy image. In: Proc. Int. Conf. Pattern recognit. (ICPR), pp 444–449Google Scholar
  81. 81.
    Yu S, et al. (2006) A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In: Proc. Int. Conf. Pattern recognit. (ICPR), vol 4, pp 441–444Google Scholar
  82. 82.
    Zeng W, Wang C, Yang F (2014) Silhouette-based gait recognition via deterministic learning. Pattern Recognit 47(11):3568–3584Google Scholar
  83. 83.
    Zhang E, Zhao Y, Xiong W (2010) Active energy image plus 2dlpp for gait recognition. Signal Process 90(7):2295–2302zbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Muhammad Hassan Khan
    • 1
    • 2
    Email author
  • Muhammad Shahid Farid
    • 2
  • Marcin Grzegorzek
    • 3
  1. 1.Research Group for Pattern RecognitionUniversity of SiegenSiegenGermany
  2. 2.Punjab University College of Information TechnologyUniversity of the PunjabPunjabPakistan
  3. 3.Institute of Medical InformaticsUniversity of LübeckLübeckGermany

Personalised recommendations