A mosaic method for multi-temporal data registration by using convolutional neural networks for forestry remote sensing applications

Abstract

Image registration is one of the most important processes for the generation of remote sensing image mosaics. This paper focuses on the special problems related to remote sensing data registration, and multi-temporal data mosaic applications in the domain of forestry. It proposes an image registration method based on hierarchical convolutional features, and applies it to improve the efficiency of large scale forestry image mosaic generation. This method uses a deep learning architecture to adaptively obtain image features from deep convolutional neural networks. The features derived from different images at different depth are sent to a correlation filter to compute the similarity between them; then the locations of the feature points are computed precisely. Based on this method, we study forestry image registration and the mosaic framework. We apply our approach to remote sensing images under different weather and seasonal conditions, and compare the results with those generated using the traditional SIFT image mosaic method. The experimental result shows that our method can detect and match the image feature points with significant spectral difference, and effectively extract feature points to generate accurate image registration and mosaic results. This demonstrates the effectiveness and robustness of the proposed approach.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. 1.

    Huang F, Zhou J, Tao J et al (2016) PMODTRAN: a parallel implementation based on MODTRAN for massive remote sensing data processing. Int J Digit Earth 9(9):819–834

    Article  Google Scholar 

  2. 2.

    Huang F et al (2017) Research on the parallelization of the DBSCAN clustering algorithm for spatial data mining based on the Spark platform. Remote Sens 9(12):1301

    Article  Google Scholar 

  3. 3.

    Li M, Liew SC, Kwoh LK (2003) Producing cloud free and cloud-shadow free mosaic from cloudy IKONOS images. In: Geoscience and remote sensing symposium, Toulouse, France, 21–25 July 2003

  4. 4.

    Liang Y, Huang WJ (2014) Detection of internal leaf structure deterioration using a new spectral ratio index in the near-infrared shoulder region. J Integr Agric 13:760–769

    Article  Google Scholar 

  5. 5.

    Qin X, Hong X, Wang H (2016) Improved SURF feature matching algorithm for remote sensing images. J Chin Mini Micro Comput Syst 37:327–331

    Google Scholar 

  6. 6.

    Xu Z, Tian Q, Zhang L (2006) Discussion on image mosaic method. Microcomput Inf 30:255–256

    Google Scholar 

  7. 7.

    Harris C, Stephens MA (1988) A combined corner and edge detector. In: Proceedings of the 4th Alvey vision conference, Manchester, UK, 31 Aug–2 Sept 1988

  8. 8.

    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110

    Article  Google Scholar 

  9. 9.

    Bay H, Ess A (2008) URF: speeded up robust features. Comput Vis Image Underst 110:346–359

    Article  Google Scholar 

  10. 10.

    Bentoutou Y, Taleb N, Kpalma K (2005) An automatic image registration for applications in remote sensing. IEEE Trans Geosci Remote Sens 43:2127–2137

    Article  Google Scholar 

  11. 11.

    Zhao Y (2012) Automatic stitching algorithm for multiple images. Comput Eng Appl 48:152–157

    Google Scholar 

  12. 12.

    Ma C, Huang J, Yang X (2015) Hierarchical convolutional features for visual tracking. In: IEEE international conference on computer vision, Chile, 13–16 Dec 2015

  13. 13.

    Cheng PM, Malhi HS (2017) Transfer learning with convolutional neural networks for classification of abdominal ultrasound images. J Digit Imaging 30:234

    Article  Google Scholar 

  14. 14.

    Boddeti V, Kanade T, Kumar B (2013) Correlation filters for object alignment. In: IEEE conference on computer vision and pattern recognition, Portland, Oregon, 25–27 June 2013

  15. 15.

    Rodriguez A, Boddeti V, Kumar B (2013) Maximum margin correlation filter: a new approach for localization and classification. IEEE Trans Image Process 22:631–643

    MathSciNet  Article  Google Scholar 

  16. 16.

    Casasent D, Ravichandran G (1992) Advanced distortion-invariant minimum average correlation energy (MACE) filters. Appl Opt 31:1109–1116

    Article  Google Scholar 

  17. 17.

    Kumar B, Mahalanobis A, Carlson D (1994) Optimal trade-off synthetic discriminant function filters for arbitrary devices. Opt Lett 19:1556–1558

    Article  Google Scholar 

  18. 18.

    Casasent D, Ravichandran G, Bollapragada S (1987) Gaussian–minimum average correlation energy filters. Appl Opt 26:3633–3640

    Article  Google Scholar 

  19. 19.

    Zhou L, Wang H (2013) Iterative adaptive synthetic correlation output filters. Electron Lett 49:878–880

    Article  Google Scholar 

  20. 20.

    Cheng Y, Zhou Y, Lin HT (2008) Automatic registration and mosaic of remote sensing images based on SIFT features. Remote Sens Technol Appl 6:721–728

    Google Scholar 

  21. 21.

    Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations, San Diego, the USA, 7–9 May 2015

  22. 22.

    Russakovsky O, Deng J, Su H (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115:211–252

    MathSciNet  Article  Google Scholar 

  23. 23.

    Chai T, Draxler RR (2014) Root mean square error (RMSE) or mean absolute error (MAE)? Geosci Model Dev 7:1525–1534

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (No. 61731022), the National Key Programme of Research and Development, Ministry of Science and Technology (No. 2017YFD0600900), and National Natural Science Foundation of China (No. 31872240). The authors would express their great appreciation to the contribution and support.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Peng Liu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zeng, Y., Ning, Z., Liu, P. et al. A mosaic method for multi-temporal data registration by using convolutional neural networks for forestry remote sensing applications. Computing 102, 795–811 (2020). https://doi.org/10.1007/s00607-019-00716-5

Download citation

Keywords

  • Data mosaic
  • Deep convolutional neural networks
  • Image registration
  • Hierarchical convolutional features
  • Forestry remote sensing