Abstract
Cross-spectral image patch matching is a challenging problem due to the significant difference between two images caused by different imaging mechanisms. We consider cross-spectral image patches can be matched because there exists a shared semantic feature space among them, in which the semantic features from different spectral images will be more independent of the spectral domains. To learn this shared feature space, we propose a progressive comparison of spatially connected feature metric learning with a feature discrimination constrain (SCFDM). The progressive comparison of spatially connected feature network keeps the property of each spectral domain in its corresponding low-level feature space, and interacts the cross-spectral features in a high-level feature space. The feature discrimination constrain enforces the framework to refine the shared semantic space for feature extraction. Extensive experiments shows that SCFDM outperforms the state-of-the-art methods on the cross-spectral dataset in terms of FPR95 and the training convergence. Meanwhile, it also demonstrates a better generalizability on a single spectral dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aguilera, C.A., Aguilera, F.J., Sappa, A.D., Aguilera, C., Toledo, R.: Learning cross-spectral similarity measures with deep convolutional neural networks. In: CVPR, pp. 1–9 (2016)
Aguilera, C.A., Sappa, A.D., Aguilera, C., Toledo, R.: Cross-spectral local descriptors via quadruplet network. Sensors 17(4), 873 (2017)
Balntas, V., Johns, E., Tang, L., Mikolajczyk, K.: PN-Net: conjoined triple deep network for learning local image descriptors. Preprint arXiv:1601.05030 (2016)
Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)
Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE TPAMI 24(4), 509–522 (2002)
Brown, M., Hua, G., Winder, S.: Discriminative learning of local image descriptors. IEEE TPAMI 33(1), 43–57 (2011)
Brown, M., Susstrunk, S.: Multi-spectral sift for scene category recognition. In: CVPR, pp. 177–184 (2011)
Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: binary robust independent elementary features. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 778–792. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15561-1_56
Chopra, S., Hadsell, R., Lecun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: CVPR, pp. 539–546 (2005)
Firmenichy, D., Brown, M., Ssstrunk, S.: Multispectral interest points for RGB-NIR image registration. In: ICIP, pp. 181–184 (2011)
Han, X., Leung, T., Jia, Y., Sukthankar, R., Berg, A.C.: MatchNet: unifying feature and metric learning for patch-based matching. In: CVPR, pp. 3279–3286 (2015)
Juefeixu, F., Pal, D.K., Savvides, M.: NIR-VIS heterogeneous face recognition via cross-spectral joint dictionary learning and reconstruction. In: CVPR, pp. 141–150 (2015)
Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: CVPR, pp. 1646–1654 (2016)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. Computer Science (2014)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: NIPS, pp. 1097–1105 (2012)
Kumar, B.G., Carneiro, G., Reid, I.: Learning local image descriptors with deep siamese and triplet convolutional networks by minimising global loss functions. In: CVPR, pp. 5385–5394 (2016)
Lin, M., Chen, Q., Yan, S.C.: Network in network. Preprint arXiv:1312.4400 (2013)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)
Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE TPAMI 27(10), 1615–1630 (2005)
Nie, F., Huang, H., Cai, X., Ding, C.H.: Efficient and robust feature selection via joint \(\ell 2\), 1-norms minimization. In: Advances in Neural Information Processing Systems, pp. 1813–1821 (2010)
Pinggera, P., Breckon, T., Bischof, H.: On cross-spectral stereo matching using dense gradient features. In: CVPR (2012)
Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: ICCV, pp. 2564–2571 (2012)
Zagoruyko, S., Komodakis, N.: Learning to compare image patches via convolutional neural networks. In: CVPR, pp. 4353–4361 (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. preprint arXiv:1409.1556 (2014)
Simoserra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., Morenonoguer, F.: Discriminative learning of deep convolutional feature point descriptors. In: ICCV, pp. 118–126 (2015)
Tian, Y., Fan, B., Wu, F.: L2-Net: deep learning of discriminative patch descriptor in Euclidean space. In: CVPR (2017)
Tola, E., Lepetit, V., Fua, P.: Daisy: an efficient dense descriptor applied to wide-baseline stereo. IEEE TPAMI 32(5), 815–830 (2010)
Wang, S., Quan, D., Liang, X., Ning, M., Guo, Y., Jiao, L.: A deep learning framework for remote sensing image registration. ISPRS J. Photogrammetry Remote Sens. 145, 148 (2018)
Winder, S., Hua, G., Brown, M.: Picking the best daisy. In: CVPR, pp. 178–185 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Quan, D., Fang, S., Liang, X., Wang, S., Jiao, L. (2019). Cross-Spectral Image Patch Matching by Learning Features of the Spatially Connected Patches in a Shared Space. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11362. Springer, Cham. https://doi.org/10.1007/978-3-030-20890-5_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-20890-5_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20889-9
Online ISBN: 978-3-030-20890-5
eBook Packages: Computer ScienceComputer Science (R0)