Abstract
This chapter introduces and analyzes a method for registering multimodal images with occluding objects in the scene. An analysis of multimodal image registration gives insight into the limitations of assumptions made in current approaches and motivates the methodology of the developed algorithm. Using calibrated stereo imagery, we use maximization of mutual information in sliding correspondence windows that inform a disparity voting algorithm to demonstrate successful registration of objects in color and thermal imagery where there is significant occlusion. Extensive testing of scenes with multiple objects at different depths and levels of occlusion shows high rates of successful registration. Ground truth experiments demonstrate the utility of disparity voting techniques for multimodal registration by yielding qualitative and quantitative results that outperform approaches that do not consider occlusions. A framework for tracking with the registered multimodal features is also presented and experimentally validated.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Chapter's References
Bertozzi, M., Broggi, A., Felias, M., Vezzoni, G., Rose, M.D. (2006) Low-level pedestrian detection by means of visible and far infra-red tetra-vision. In: IEEE Conference on Intelligent Vehicles
Davis, J., Sharma, V. (2005) Fusion-based background-subtraction using contour saliency. In: IEEE CVPR Workshop on Object Tracking and Classification beyond the Visible Spectrum
Egnal, G. (2000) Mutual information as a stereo correspondence measure. Technical Report MS-CIS-00-20, University of Pennsylvania
Chen, H., Varshney, P., Slamani, M. (2003) On registration of regions of interest (ROI) in video sequences. In: IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS'03), pp. 313
Conaire, C.O., Cooke, E., O'Connor, N., Murphy, N., Smeaton, A. (2005) Background modeling in infrared and visible spectrum video for people tracking. In: IEEE CVPR Workshop on Object Tracking and Classification beyond the Visible Spectrum
Han, J., Bhanu, B. (2003) Detecting moving humans using color and infrared video. In: IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems
Itoh, M., Ozeki, M., Nakamura, Y., Ohta, Y. (2003) Simple and robust tracking of hands and objects for video-based multimedia production. In: IEEE Conference on Multisensor Fusion and Integration for Intelligent Systems
Irani, M., Anandan, P. (1998) Robust multi-sensor image alignment. In: Sixth International Conference on Computer Vision, 1998
Coiras, E., Santamaria, J., Miravet, C. (2000) Segment-based registration technique for visual-infrared images. Optical Engineering 39(1), 282–289
Chen, H., Lee, S., Rao, R., Slamani, M., Varshney, P. (2005) Imaging for concealed weapon detection. Signal Processing Magazine, IEEE Vol. 22, Issue 2, March 2005, pp. 52–61
Kim, J., Kolmogorov, V., Zabih, R. (2003) Visual correspondence using energy minimization and mutual information. In: Ninth IEEE International Conference on Computer Vision
H. Hirschmüller (2005) Accurate and efficient stereo processing by semi-global matching and mutual information. In: Computer Vision and Pattern Recognition
Scharstein, D., Szeliski, R. (2005) Middlebury College stereo vision research page. http://bj.middlebury.edu/?schar/stereo/web/results.php
Marapane, S., Trivedi, M. (1989) Region-based stereo analysis for robotic applications. IEEE Transactions on Systems, Man, and Cybernetics, Special Issue on Computer Vision 19(6), 1447–1464
Cohen, L., Vinet, L., Sander, P., Gagalowicz, A. (1989) Hierarchical region based stereo matching. In: Computer Vision and Pattern Recognition
Wei, Y., Quan, L. (2004) Region-based progressive stereo matching. In: Computer Vision and Pattern Recognition
Bleyer, M., Gelautz, M. (2005) Graph-based surface reconstruction from stereo pairs using image segmentation. Proc. SPIE 5665, 288–299
Krotosky, S.J., Trivedi, M.M. (2006) Registration of multimodal stereo images using disparity voting from correspondence windows. In: IEEE Conference on Advanced Video and Signal based Surveillance (AVSS'06)
Thevenaz, P., Unser, M. (2000) Optimization of mutual information for multiresolution image registration. IEEE Transactions on Image Processing 9(12), 2083–2089
Kim, K., Chalidabhongse, T., Harwood, D., Davis, L. (2005) Real-time foreground-background segmentation using codebook model. Real-Time Imaging 11(3), 163–256
Moesland, T.B., Granum, E. (2001) A survey of computer vision-based human motion capture. Computer Vision and Image Understanding 81(3), 231–268
Harville, M., Li, D. (2004) Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera. In: IEEE Conference on Computer Vision and Pattern Recognition
Huang, K., Trivedi, M.M. (2003) Video arrays for real-time tracking of person, head, and face in an intelligent room. Machine Vision and Applications 14(2), 103–111
Marapane, S., Trivedi, M.M. (1994) Multi-primitive hierarchical (MPH) stereo analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 16(3), 227–240
Thevenaz, P., Bierlaire, M., Unser, M.: Halton sampling for image registration based on mutual information. Sampling Theory in Signal and Image Processing, May, 2008
Trivedi, M.M., Cheng, S.Y., Childers, E.M.C., Krotosky, S.J. (2004) Occupant posture analysis with stereo and thermal infrared video: Algorithms and experimental evaluation. IEEE Transactions on Vehicle Technology 53(6), 1968–1712
Krotosky, S.J., Trivedi, M.M. (2006) Multimodal stereo image registration for predestrian detection. In: IEEE Conference on Intelligent Transportation Systems
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Krotosky, S., Trivedi, M. (2009). Registering Multimodal Imagery with Occluding Objects Using Mutual Information: Application to Stereo Tracking of Humans. In: Hammoud, R.I. (eds) Augmented Vision Perception in Infrared. Advances in Pattern Recognition. Springer, London. https://doi.org/10.1007/978-1-84800-277-7_14
Download citation
DOI: https://doi.org/10.1007/978-1-84800-277-7_14
Publisher Name: Springer, London
Print ISBN: 978-1-84800-276-0
Online ISBN: 978-1-84800-277-7
eBook Packages: Computer ScienceComputer Science (R0)