Advertisement

Fusion of Color/Infrared Video for Human Detection

  • Bir Bhanu
  • Ju Han
Part of the Advances in Pattern Recognition book series (ACVPR)

Abstract

In this chapter, we approach the task of human silhouette extraction from color and thermal image sequences using automatic image registration. Image registration between color and thermal images is a challenging problem due to the difficulties associated with finding correspondence. However, moving people in a static scene provide cues to address this problem. We first propose a hierarchical scheme to automatically find the correspondence between the preliminary human silhouettes extracted from synchronous color and thermal image sequences for image registration. Next, we discuss strategies for probabilistically combining cues from registered color and thermal images for improved human silhouette detection. It is shown that the proposed approach achieves good results for image registration and human silhouette extraction. Experimental results also show a comparison of various sensor fusion strategies and demonstrate the improvement in performance over non-fused cases for human silhouette extraction.

Keywords

Image Registration Thermal Image Search Level Automatic Image Registration Human Silhouette 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 4.
    Ali, M.A., Clausi, D.A.: Automatic registration of SAR and visible band remote sensing images. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium, vol. 3, pp. 1331–1333 (2002) Google Scholar
  2. 6.
    Arlowe, H.D.: Thermal detection contrast of human targets. In: Proceedings of IEEE International Carnahan Conference on Security Technology, pp. 27–33 (1992) Google Scholar
  3. 14.
    Bhanu, B., Han, J.: Kinematic-based human motion analysis in infrared sequences. In: Proceedings of IEEE Workshop on Applications of Computer Vision, pp. 208–212 (2002) Google Scholar
  4. 31.
    Clark, G.A., Sengupta, S.K., Buhl, M.R., Sherwood, R.J., Schaich, P.C., Bull, N., Kane, R.J., Barth, M.J., Fields, D.J., Carter, M.R.: Detecting buried objects by fusing dual-band infrared images. In: 1993 Conference Record of the Twenty-Seventh Asilomar Conference on Signals, Systems and Computers, vol. 1, pp. 135–143 (1993) Google Scholar
  5. 37.
    Dare, P.M., Dowman, I.J.: Automatic registration of SAR and spot imagery based on multiple feature extraction and matching. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium, vol. 7, pp. 2896–2898 (2000) Google Scholar
  6. 62.
    Han, J., Bhanu, B.: Moving human detection by EO and IR sensor fusion. Pattern Recognit. 40(6), 1771–1784 (2007) zbMATHCrossRefGoogle Scholar
  7. 72.
    Huang, P.S., Harris, C.J., Nixon, M.S.: Recognizing humans by gait via parametric canonical space. Artif. Intell. Eng. 13, 359–366 (1999) CrossRefGoogle Scholar
  8. 74.
    Inglada, J., Adragna, F.: Automatic multi-sensor image registration by edge matching using genetic algorithms. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium, vol. 5, pp. 2313–2315 (2001) Google Scholar
  9. 82.
    Kale, A., Rajagopalan, A.N., Cuntoor, N., Kruger, V.: Gait-based recognition of humans using continuous HMMs. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 321–326 (2002) Google Scholar
  10. 88.
    Kittler, J., Hatef, M., Duin, R., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998) CrossRefGoogle Scholar
  11. 100.
    Li, H., Zhou, Y.T.: Automatic EO/IR sensor image registration. In: Proceedings of the International Conference on Image Processing, vol. 3, pp. 240–243 (1995) Google Scholar
  12. 101.
    Li, H., Manjunath, B.S., Mitra, S.K.: A contour-based approach to multisensor image registration. IEEE Trans. Image Process. 4(3), 320–334 (1995) CrossRefGoogle Scholar
  13. 102.
    Li, H., Zhou, Y.T., Chellappa, R.: SAR/IR sensor image fusion and real-time implementation. In: Record of the Twenty-Ninth Asilomar Conference on Signals, Systems and Computers, vol. 2, pp. 1121–1125 (1995) Google Scholar
  14. 107.
    Little, J.J., Boyd, J.E.: Recognizing people by their gait: the shape of motion. Videre, J. Comput. Vis. Res. 1(2), 1–32 (1998) Google Scholar
  15. 115.
    Mandava, V.R., Fitzpatrick, J.M., Pickens, D.R.I.: Adaptive search space scaling in digital image registration. IEEE Trans. Med. Imaging 8(3), 251–262 (1989) CrossRefGoogle Scholar
  16. 120.
    Nadimi, S., Bhanu, B.: Multistrategy fusion using mixture model for moving object detection. In: Proceedings of International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp. 317–322 (2001) Google Scholar
  17. 121.
    Nadimi, S., Bhanu, B.: Physical models for moving shadow and object detection in video. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 1079–1087 (2004) CrossRefGoogle Scholar
  18. 122.
    Nanda, H., Davis, L.: Probabilistic template based pedestrian detection in infrared videos. In: Proceedings of IEEE Intelligent Vehicle Symposium, vol. 1, pp. 15–20 (2002) Google Scholar
  19. 126.
    Niyogi, S.A., Adelson, E.H.: Analyzing and recognizing walking figures in XYT. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 469–474 (1994) Google Scholar
  20. 133.
    Perez-Jacome, J.E., Madisetti, V.K.: Target detection from coregistered visual-thermal-range images. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 4, pp. 2741–2744 (1997) Google Scholar
  21. 137.
    Phillips, P.J., Sarkar, S., Robledo, I., Grother, P., Bowyer, K.: The gait identification challenge problem: data sets and baseline algorithm. In: Proceedings of International Conference on Pattern Recognition, vol. 1, pp. 385–388 (2002) Google Scholar
  22. 148.
    Santiago-Mozos, R., Leiva-Murillo, J.M., Perez-Cruz, F., Artes-Rodriguez, A.: Supervised-PCA and SVM classifiers for object detection in infrared images. In: Proceedings of IEEE Conference on Advanced Video and Signal Based Surveillance, pp. 122–127 (2003) Google Scholar
  23. 166.
    Tolliver, D., Collins, R.T.: Gait shape estimation for identification. In: Proceedings of fourth International Conference on Audio- and Video-Based Biometric Person Authentication. LNCS, vol. 2688, pp. 734–742 (2003) Google Scholar
  24. 171.
    van den Elsen, P.A., Pol, E.-J.D., Viergever, M.A.: Medical image matching—a review with classification. IEEE Eng. Med. Biol. Mag. 12(1), 26–39 (1993) CrossRefGoogle Scholar
  25. 183.
    Wilder, J., Phillips, P.J., Jiang, C., Wiener, S.: Comparison of visible and infra-red imagery for face recognition. In: Proceedings of International Conference on Automatic Face and Gesture Recognition, pp. 182–187 (1996) Google Scholar
  26. 192.
    Yao, J.: Image registration based on both feature and intensity matching. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 3, pp. 1693–1696 (2001) Google Scholar
  27. 194.
    Yoshitomi, Y., Kim, S.-I., Kawano, T., Kilazoe, T.: Effect of sensor fusion for recognition of emotional states using voice, face image and thermal image of face. In: Proceedings of IEEE International Workshop on Robot and Human Interactive Communication, pp. 178–183 (1996) Google Scholar
  28. 206.
    Zheng, Q., Chellappa, R.: A computational vision approach to image registration. IEEE Trans. Image Process. 2(3), 311–326 (1993) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2010

Authors and Affiliations

  1. 1.Bourns College of EngineeringUniversity of CaliforniaRiversideUSA
  2. 2.Lawrence Berkeley National LaboratoryUniversity of CaliforniaBerkeleyUSA

Personalised recommendations