Feature Level Fusion of Face and Gait at a Distance

  • Bir Bhanu
  • Ju Han
Part of the Advances in Pattern Recognition book series (ACVPR)


Video-based human recognition at a distance remains a challenging problem for the fusion of multi-modal biometrics. As compared to the approach based on match score level fusion (Chap. 10), in this chapter, we present an approach that utilizes and integrates information from side face and gait at the feature level. The features of face and gait are obtained separately using principal component analysis (PCA) from enhanced side face image (ESFI) and gait energy image (GEI), respectively. Multiple discriminant analysis (MDA) is employed on the concatenated features of face and gait to obtain discriminating synthetic features. This process allows the generation of better features and reduces the curse of dimensionality. The proposed scheme is tested using two comparative data sets to show the effect of changing clothes and face changing over time. Moreover, the proposed feature level fusion is compared with the match score level fusion and another feature level fusion scheme. The experimental results demonstrate that the synthetic features, encoding both side face and gait information, carry more discriminating power than the individual biometrics features, and the proposed feature level fusion scheme outperforms the match score level and another feature level fusion scheme. The performance of different fusion schemes is also shown as cumulative match characteristic (CMC) curves. They further demonstrate the strength of the proposed fusion scheme.


Covariance Shoe 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 16.
    Bhanu, B., Zhou, X.: Face recognition from face profile using dynamic time warping. In: Proceedings of International Conference on Pattern Recognition, vol. 4, pp. 499–502 (2004) Google Scholar
  2. 45.
    Feng, G., Dong, K., Hu, D., Zhang, D.: When faces are combined with palmprints: a novel biometric fusion strategy. In: International Conference on Biometric Authentication, pp. 701–707 (2004) Google Scholar
  3. 49.
    Gao, Y., Maggs, M.: Feature-level fusion in personal identification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2005) Google Scholar
  4. 51.
    Ginesu, G., Giusto, D.D., Margner, V., Meinlschmidt, P.: Digital Image Processing. Addison-Wessley, Reading (1992) Google Scholar
  5. 61.
    Han, J., Bhanu, B.: Individual recognition using gait energy image. IEEE Trans. Pattern Anal. Mach. Intell. 28(2), 316–322 (2006) CrossRefGoogle Scholar
  6. 84.
    Kale, A., Roy-chowdhury, A., Chellappa, R.: Fusion of gait and face for human identification. In: Proceedings of Acoustics, Speech, and Signal Processing, vol. 5, pp. 901–904 (2004) Google Scholar
  7. 87.
    Kinnunen, T., Hautamaki, V., Franti, P.: Fusion of spectral feature sets for accurate speaker identification. In: Proceedings of International Conference Speech and Computer, pp. 361–365 (2004) Google Scholar
  8. 89.
    Kong, A., Zhang, D., Kamel, M.: Palmprint identification using feature-level fusion. Pattern Recognit. 39(3), 478–487 (2006) MATHCrossRefGoogle Scholar
  9. 93.
    Kumar, A., Wong, D.C.M., Shen, H.C., Jain, A.K.: Personal verification using palmprint and hand geometry biometric. In: Proceedings of Audio- and Video-Based Biometric Person Authentication, pp. 668–678 (2003) Google Scholar
  10. 99.
    Li, Q., Qiu, Z.: Handmetric verification based on feature-level fusion. J. Comput. Sci. Netw. Secur. 6(2A) (2006) Google Scholar
  11. 117.
    Moon, Y.S., Yeung, H.W., Chan, K.C., Chan, S.O.: Template synthesis and image mosaicking for fingerprint registration: an experimental study. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 5, pp. 409–412 (2004) Google Scholar
  12. 145.
    Ross, A.A., Govindarajan, R.: Feature level fusion of hand and face biometrics. In: Proceedings of SPIE Conference on Biometric Technology for Human Identification II, pp. 196–204 (2006) Google Scholar
  13. 146.
    Ross, A.A., Nandakumar, K., Jain, A.K.: Handbook of multibiometrics. In: Proceedings of IEEE International Conference on Image Processing (2006) Google Scholar
  14. 153.
    Shakhnarovich, G., Darrell, T.: On probabilistic combination of face and gait cues for identification. In: Proceedings of Automatic Face and Gesture Recognition, vol. 5, pp. 169–174 (2002) Google Scholar
  15. 154.
    Shakhnarovich, G., Lee, L., Darrell, T.: Integrated face and gait recognition from multiple views. In: Proceedings of IEEE Workshop on Visual Motion, vol. 1, pp. 439–446 (2001) Google Scholar
  16. 163.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition. Academic Press, San Diego (1998) MATHGoogle Scholar
  17. 189.
    Yambor, W.S., Draper, B.A., Beveridge, J.R.: Analyzing PCA-Based Face Recognition Algorithms: Eigenvector Selection and Distance Measures. World Scientific, Singapore (2002) Google Scholar
  18. 191.
    Yang, J., Yang, J.Y., Zhang, D., Lu, J.F.: Feature fusion: parallel strategy vs. serial strategy. Pattern Recognit. 38(6), 1369–1381 (2003) CrossRefGoogle Scholar
  19. 207.
    Zhou, X., Bhanu, B.: Feature fusion of face and gait for human recognition at a distance in video. In: Proceedings of IEEE International Conference Pattern Recognition (2006) Google Scholar
  20. 208.
    Zhou, X., Bhanu, B.: Integrating face and gait for human recognition. In: Proceedings of Workshop on Biometrics Held in Conjunction with the IEEE Conference Computer Vision and Pattern Recognition, pp. 55–62 (2006) Google Scholar
  21. 209.
    Zhou, X., Bhanu, B.: Integrating face and gait for human recognition at a distance in video. IEEE Trans. Syst. Man Cybern., Part B, Cybern. 37, 55–62 (2007) CrossRefGoogle Scholar
  22. 211.
    Zhou, X., Bhanu, B., Han, J.: Human recognition at a distance in video by integrating face profile and gait. In: Proceedings of Audio- and Video-Based Biometric Person Authentication, pp. 533–543 (2005) Google Scholar

Copyright information

© Springer-Verlag London Limited 2010

Authors and Affiliations

  1. 1.Bourns College of EngineeringUniversity of CaliforniaRiversideUSA
  2. 2.Lawrence Berkeley National LaboratoryUniversity of CaliforniaBerkeleyUSA

Personalised recommendations