The Journal of Supercomputing

, Volume 75, Issue 3, pp 1551–1564 | Cite as

A multi-device version of the HYFMGPU algorithm for hyperspectral scenes registration

  • Jorge Fernández-FabeiroEmail author
  • Álvaro Ordóñez
  • Arturo Gonzalez-Escribano
  • Dora B. Heras


Hyperspectral image registration is a relevant task for real-time applications like environmental disasters management or search and rescue scenarios. Traditional algorithms were not really devoted to real-time performance, even when ported to GPUs or other parallel devices. Thus, the HYFMGPU algorithm arose as a solution to such a lack. Nevertheless, as sensors are expected to evolve and thus generate images with finer resolutions and wider wavelength ranges, a multi-GPU implementation of this algorithm seems to be necessary in a near future. This work presents a multi-device MPI \(+\) CUDA implementation of the HYFMGPU algorithm that distributes all its stages among several GPUs. This version has been validated testing it for 5 different real hyperspectral images, with sizes from about 80 MB to nearly 2 GB, achieving speedups for the whole execution of the algorithm from 1.18 \(\times \) to 1.59 \(\times \) in 2 GPUs and from 1.26 \(\times \) to 2.58 \(\times \) in 4 GPUs. The parallelization efficiencies obtained are stable around 86\(\%\) and 78\(\%\) for 2 and 4 GPUs, respectively, which proves the scalability of this multi-device version.


Hyperspectral imaging Image registration Fourier transforms Multi-GPU CUDA OpenMP MPI Remote sensing 



This work has been partially supported by: Universidad de Valladolid—Consejería de Educación of Junta de Castilla y León, Ministerio de Economía, Industria y Competitividad of Spain, and European Regional Development Fund (ERDF) program: Project PCAS (TIN2017-88614-R), Project PROPHET (VA082P17) and CAPAP-H6 network (TIN2016-81840-REDT). Universidade de Santiago de Compostela—Consellería de Cultura, Educación e Ordenación Universitaria of Xunta de Galicia (grant numbers GRC2014/008 and ED431G/08) and Ministerio de Economía, Industria y Competitividad of Spain (Grant Number TIN2016-76373-P), all co-funded by the European Regional Development Fund (ERDF) program. The work of Álvaro Ordóñez was supported by the Ministerio de Educación, Cultura y Deporte under an FPU Grant (Grant Number FPU16/03537).


  1. 1.
    Awan AA, Hamidouche K, Venkatesh A, Panda DK (2016) Efficient large message broadcast using NCCL and CUDA-aware MPI for deep learning. In: Proceedings of the 23rd European MPI Users’ Group Meeting, EuroMPI 2016. ACM, New York, pp. 15–22Google Scholar
  2. 2.
    Baumgardner MF, Biehl LL, Landgrebe DA (1992) 220 Band AVIRIS Hyperspectral Image Data Set: June 12, 1992 Indian Pine Test Site 3. Accessed 14 Nov 2018
  3. 3.
    Fernández-Fabeiro J, Álvaro Ordóñez, González-Escribano A, Heras DB (2018) Towards a multi-device version of the HYFMGPU algorithm for hyperspectral scenes registration.
  4. 4.
    Frigo M, Johnson SG (2005) The Design and Implementation of FFTW3. Proc IEEE 93(2):216–231CrossRefGoogle Scholar
  5. 5.
    Garea AS, Heras DB, Argüello F (2016) GPU classification of remote-sensing images using kernel ELM and extended morphological profiles. Int J Remote Sens 37(24):5918–5935CrossRefGoogle Scholar
  6. 6.
    Glaser J, Nguyen TD, Anderson JA, Lui P, Spiga F, Millan JA, Morse DC, Glotzer SC (2015) Strong scaling of general-purpose molecular dynamics simulations on GPUs. Comput Phys Commun 192:97–107CrossRefGoogle Scholar
  7. 7.
    Jiménez LI, Sánchez S, Martín G, Plaza J, Plaza AJ (2017) Parallel Implementation of spatial spectral endmember extraction on graphic processing units. IEEE J Sel Top Appl Earth Obs Remote Sens 10(4):1247–1255CrossRefGoogle Scholar
  8. 8.
    Keshava N, Mustard JF (2002) Spectral unmixing. IEEE Signal Process Mag 19(1):44–57CrossRefGoogle Scholar
  9. 9.
    Kraus J (2013) An Introduction to CUDA-Aware MPI, NVIDIA Developer Blog. Accessed 14 Nov 2018
  10. 10.
    Lončar V, Young SLE, Škrbić S, Muruganandam P, Adhikari SK, Balaž A (2016) OpenMP, OpenMP, MPI, and CUDA/MPI C programs for solving the time-dependent dipolar Gross–Pitaevskii equation. Comput Phys Commun 209:190–196CrossRefzbMATHGoogle Scholar
  11. 11.
    NASA Jet Propulsion Laboratory: Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) Database. Accessed 14 Nov 2018
  12. 12.
    NVIDIA Corporation: cuBLAS Library User’s Guide. Accessed 14 Nov 2018
  13. 13.
    NVIDIA Corporation: CUDA C Programming Guide. Accessed 14 Nov 2018
  14. 14.
    NVIDIA Corporation: cuFFT Library User’s Guide. Accessed 14 Nov 2018
  15. 15.
    NVIDIA Corporation: cuSOLVER Library User’s Guide. Accessed 14 Nov 2018
  16. 16.
    NVIDIA Corporation: MPI Solutions for GPUs. Accessed 14 Nov 2018
  17. 17.
    NVIDIA Corporation: NVIDIA Collective Communications Library (NCCL). Accessed 14 Nov 2018
  18. 18.
    NVIDIA Corporation: NVIDIA Performance Primitives (NPP). Accessed 14 Nov 2018
  19. 19.
    NVIDIA Corporation: Thrust Quick Start Guide. Accessed 14 Nov 2018
  20. 20.
    Ordóñez A, Argüello F, Heras DB Repository of hyperspectral images for ’GPU Accelerated FFT-Based Registration of Hyperspectral Scenes’. Accessed 14 Nov 2018
  21. 21.
    Ordóñez A, Argüello F, Heras DB (2017) Fourier–Mellin registration of two hyperspectral images. Int J Remote Sens 38(11):3253–3273CrossRefGoogle Scholar
  22. 22.
    Ordóñez A, Argüello F, Heras DB (2017) GPU accelerated FFT-based registration of hyperspectral scenes. IEEE J Sel Top Appl Earth Obs Remote Sens 10(11):4869–4878CrossRefGoogle Scholar
  23. 23.
    Pan W, Qin K, Chen Y (2009) An adaptable-multilayer fractional Fourier transform approach for image registration. IEEE Trans Pattern Anal Mach Intell 31(3):400–414CrossRefGoogle Scholar
  24. 24.
    Richards J (2013) Remote sensing digital image analysis, chap. Feature reduction. Springer, Berlin, pp 343–380CrossRefGoogle Scholar
  25. 25.
    Rogge D, Rivard B, Zhang J, Sanchez A, Harris J, Feng J (2007) Integration of spatial–spectral information for the improved extraction of endmembers. Remote Sens Environ 110(3):287–303CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Departamento de InformáticaUniversidad de ValladolidValladolidSpain
  2. 2.Centro Singular de Investigación en Tecnoloxías da Información (CiTIUS)Universidade de Santiago de CompostelaSantiago de CompostelaSpain

Personalised recommendations