A Fusion Approach to Grayscale-Thermal Tracking with Cross-Modal Sparse Representation
Grayscale-thermal tracking receives much attention recently due to the complementary benefits of the visible and thermal infrared modalities in over- coming the imaging limitations of individual source. This paper investigates how to perform effective fusion of the grayscale and thermal information for robust object tracking. We propose a novel fusion approach based on the cross-modal sparse representation in the Bayesian filtering framework. First, to exploit the interdependence of different modalities, we take both the intra- and inter-modality constraints into account in the sparse representation, i.e., cross-modal sparse rep- resentation. Moreover, we introduce the modality weights in our model to achieve adaptive fusion. Second, unlike conventional methods, we employ the reconstruction residues and coefficients together to define the likelihood probability for each candidate sample generated by the motion model. Finally, the object is located by finding the candidate sample with the maximum likelihood probability. Experimental results on the public benchmark dataset suggest that the proposed approach performs favourably against the state-of-the-art grayscale-thermal trackers.
KeywordsMulti-modal Fusion Laplacian matrix Sparse representation Bayesian filtering
This work was supported in part by the Natural Science Foundation of Anhui Higher Education Institution of China under Grants KJ2017A017, and in part by the Co- Innovation Center for Information Supply & Assurance Technology, Anhui University under Grant Y01002449.
- 1.Bunyak, F., Palaniappan, K., Nath, S.K., Seetharaman, G.: Geodesic active contour based fusion of visible and infrared video for persistent object tracking. In: Proceedings of IEEE Workshop on Applications of Computer Vision (2007)Google Scholar
- 2.Conaire, C.O., Connor, N.E., Cooke, E., Smeaton, A.F.: Comparison of fusion methods for thermo-visual surveillance tracking. In: Proceedings of International Conference on Information Fusion (2006)Google Scholar
- 3.Conaire, C.O., Connor, N.E., Smeaton, A.: Thermo-visual feature fusion for object tracking using multiple spatiogram trackers. Mach. Vis. Appl. 7, 1–12 (2007)Google Scholar
- 4.Cvejic, N., et al.: The effect of pixel-level fusion on object tracking in multi-sensor surveillance video. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2007)Google Scholar
- 6.Hare, S., Saffari, A., Torr, P.H.S.: Struck: structured output tracking with kernels. In: Proceedings of IEEE International Conference on Computer Vision (2011)Google Scholar
- 12.Mei, X., Ling, H.: Robust visual tracking using l1 minimization. In: Proceedings of IEEE International Conference on Computer Vision (2009)Google Scholar
- 13.Wu, Y., Blasch, E., Chen, G., Bai, L., Ling, H.: Multiple source data fusion via sparse representation for robust visual tracking. In: Proceedings of International Conference on Information Fusion (2011)Google Scholar
- 16.Zhang, T., Ghanem, B., Liu, S., Ahuja, N.: Robust visual tracking via multi-task sparse learning. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2012)Google Scholar
- 17.Zhong, W., Lu, H., Yang, M.H.: Robust object tracking via sparsity-based collaborative model. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2012)Google Scholar