pp 1–17 | Cite as

A mosaic method for multi-temporal data registration by using convolutional neural networks for forestry remote sensing applications

  • Yi Zeng
  • Zihan Ning
  • Peng LiuEmail author
  • Peilei Luo
  • Yi Zhang
  • Guojin He


Image registration is one of the most important processes for the generation of remote sensing image mosaics. This paper focuses on the special problems related to remote sensing data registration, and multi-temporal data mosaic applications in the domain of forestry. It proposes an image registration method based on hierarchical convolutional features, and applies it to improve the efficiency of large scale forestry image mosaic generation. This method uses a deep learning architecture to adaptively obtain image features from deep convolutional neural networks. The features derived from different images at different depth are sent to a correlation filter to compute the similarity between them; then the locations of the feature points are computed precisely. Based on this method, we study forestry image registration and the mosaic framework. We apply our approach to remote sensing images under different weather and seasonal conditions, and compare the results with those generated using the traditional SIFT image mosaic method. The experimental result shows that our method can detect and match the image feature points with significant spectral difference, and effectively extract feature points to generate accurate image registration and mosaic results. This demonstrates the effectiveness and robustness of the proposed approach.


Data mosaic Deep convolutional neural networks Image registration Hierarchical convolutional features Forestry remote sensing 



This work was supported by National Natural Science Foundation of China (No. 61731022), the National Key Programme of Research and Development, Ministry of Science and Technology (No. 2017YFD0600900), and National Natural Science Foundation of China (No. 31872240). The authors would express their great appreciation to the contribution and support.


  1. 1.
    Huang F, Zhou J, Tao J et al (2016) PMODTRAN: a parallel implementation based on MODTRAN for massive remote sensing data processing. Int J Digit Earth 9(9):819–834CrossRefGoogle Scholar
  2. 2.
    Huang F et al (2017) Research on the parallelization of the DBSCAN clustering algorithm for spatial data mining based on the Spark platform. Remote Sens 9(12):1301CrossRefGoogle Scholar
  3. 3.
    Li M, Liew SC, Kwoh LK (2003) Producing cloud free and cloud-shadow free mosaic from cloudy IKONOS images. In: Geoscience and remote sensing symposium, Toulouse, France, 21–25 July 2003Google Scholar
  4. 4.
    Liang Y, Huang WJ (2014) Detection of internal leaf structure deterioration using a new spectral ratio index in the near-infrared shoulder region. J Integr Agric 13:760–769CrossRefGoogle Scholar
  5. 5.
    Qin X, Hong X, Wang H (2016) Improved SURF feature matching algorithm for remote sensing images. J Chin Mini Micro Comput Syst 37:327–331Google Scholar
  6. 6.
    Xu Z, Tian Q, Zhang L (2006) Discussion on image mosaic method. Microcomput Inf 30:255–256Google Scholar
  7. 7.
    Harris C, Stephens MA (1988) A combined corner and edge detector. In: Proceedings of the 4th Alvey vision conference, Manchester, UK, 31 Aug–2 Sept 1988Google Scholar
  8. 8.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110CrossRefGoogle Scholar
  9. 9.
    Bay H, Ess A (2008) URF: speeded up robust features. Comput Vis Image Underst 110:346–359CrossRefGoogle Scholar
  10. 10.
    Bentoutou Y, Taleb N, Kpalma K (2005) An automatic image registration for applications in remote sensing. IEEE Trans Geosci Remote Sens 43:2127–2137CrossRefGoogle Scholar
  11. 11.
    Zhao Y (2012) Automatic stitching algorithm for multiple images. Comput Eng Appl 48:152–157Google Scholar
  12. 12.
    Ma C, Huang J, Yang X (2015) Hierarchical convolutional features for visual tracking. In: IEEE international conference on computer vision, Chile, 13–16 Dec 2015Google Scholar
  13. 13.
    Cheng PM, Malhi HS (2017) Transfer learning with convolutional neural networks for classification of abdominal ultrasound images. J Digit Imaging 30:234CrossRefGoogle Scholar
  14. 14.
    Boddeti V, Kanade T, Kumar B (2013) Correlation filters for object alignment. In: IEEE conference on computer vision and pattern recognition, Portland, Oregon, 25–27 June 2013Google Scholar
  15. 15.
    Rodriguez A, Boddeti V, Kumar B (2013) Maximum margin correlation filter: a new approach for localization and classification. IEEE Trans Image Process 22:631–643MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Casasent D, Ravichandran G (1992) Advanced distortion-invariant minimum average correlation energy (MACE) filters. Appl Opt 31:1109–1116CrossRefGoogle Scholar
  17. 17.
    Kumar B, Mahalanobis A, Carlson D (1994) Optimal trade-off synthetic discriminant function filters for arbitrary devices. Opt Lett 19:1556–1558CrossRefGoogle Scholar
  18. 18.
    Casasent D, Ravichandran G, Bollapragada S (1987) Gaussian–minimum average correlation energy filters. Appl Opt 26:3633–3640CrossRefGoogle Scholar
  19. 19.
    Zhou L, Wang H (2013) Iterative adaptive synthetic correlation output filters. Electron Lett 49:878–880CrossRefGoogle Scholar
  20. 20.
    Cheng Y, Zhou Y, Lin HT (2008) Automatic registration and mosaic of remote sensing images based on SIFT features. Remote Sens Technol Appl 6:721–728Google Scholar
  21. 21.
    Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations, San Diego, the USA, 7–9 May 2015Google Scholar
  22. 22.
    Russakovsky O, Deng J, Su H (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115:211–252MathSciNetCrossRefGoogle Scholar
  23. 23.
    Chai T, Draxler RR (2014) Root mean square error (RMSE) or mean absolute error (MAE)? Geosci Model Dev 7:1525–1534CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Austria, part of Springer Nature 2019

Authors and Affiliations

  • Yi Zeng
    • 1
  • Zihan Ning
    • 1
  • Peng Liu
    • 2
    Email author
  • Peilei Luo
    • 1
  • Yi Zhang
    • 1
  • Guojin He
    • 2
  1. 1.School of Information Science and TechnologyBeijing Forestry UniversityBeijingChina
  2. 2.Institute of Remote Sensing and Digital EarthChinese Academy of SciencesBeijingChina

Personalised recommendations