Sharpening Hyperspectral Images Using Spatial and Spectral Priors in a Plug-and-Play Algorithm

  • Afonso M. Teodoro
  • José M. Bioucas-Dias
  • Mário A. T. FigueiredoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10746)


This paper proposes using both spatial and spectral regularizers/priors for hyperspectral image sharpening. Leveraging the recent plug-and-play framework, we plug two Gaussian-mixture-based denoisers into the iterations of an alternating direction method of multipliers (ADMM): a spatial regularizer learned from the observed multispectral image, and a spectral regularizer trained using the hyperspectral data. The proposed approach achieves very competitive results, improving the performance over using a single regularizer. Furthermore, the spectral regularizer can be used to classify the image pixels, opening the door to class-adapted models.


Data fusion Hyperspectral imaging Spatial-spectral regularization Plug-and-play Gaussian mixture 



This work was partially supported by Fundação para a Ciência e Tecnologia (FCT), grants BD/102715/2014, UID/EEA/5008/2013, and ERANETMED/0001/2014. The authors would like to thank Prof. N. Yokoya for providing the datasets [26].


  1. 1.
    Afonso, M., Bioucas-Dias, J., Figueiredo, M.: Fast image recovery using variable splitting and constrained optimization. IEEE Trans. Image Process. 19, 2345–2356 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Ssstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274–2282 (2012)CrossRefGoogle Scholar
  3. 3.
    Bauschke, H., Combettes, P.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, Heidelberg (2011). CrossRefzbMATHGoogle Scholar
  4. 4.
    Bioucas-Dias, J., Nascimento, J.: Hyperspectral subspace identification. IEEE Trans. Geosci. Remote Sens. 46, 2435–2445 (2008)CrossRefGoogle Scholar
  5. 5.
    Bioucas-Dias, J., Plaza, A., Dobigeon, N., Parente, M., Du, Q., Gader, P., Chanussot, J.: Hyperspectral unmixing overview: geometrical, statistical, and sparse regression-based approaches. IEEE J. Sel. Topics Appl. Earth Obs. Remote Sens. 5, 354–379 (2012)CrossRefGoogle Scholar
  6. 6.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2011)CrossRefzbMATHGoogle Scholar
  7. 7.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 1222–1239 (2001)CrossRefGoogle Scholar
  8. 8.
    Brifman, A., Romano, Y., Elad, M.: Turning a denoiser into a super-resolver using plug and play priors. In: IEEE ICIP (2016)Google Scholar
  9. 9.
    Chan, S., Wang, X., Elgendy, O.: Plug-and-play ADMM for image restoration: fixed point convergence and applications. IEEE Trans. Comput. Imaging PP(99), 1 (2016)Google Scholar
  10. 10.
    Green, A., Berman, M., Switzer, P., Craig, M.: A transformation for ordering multispectral data in terms of image quality with implications for noise removal. IEEE Trans. Geosci. Remote Sens. 26, 65–74 (1988)CrossRefGoogle Scholar
  11. 11.
    Jolliffe, I.: Principal Component Analysis. Springer, New York (1986). CrossRefzbMATHGoogle Scholar
  12. 12.
    Landgrebe, D.: Signal Theory Methods in Multispectral Remote Sensing. Wiley, Hoboken (2003)CrossRefGoogle Scholar
  13. 13.
    Loncan, L., Almeida, L., Bioucas-Dias, J., Briottet, X., Chanussot, J., Dobigeon, N., Fabre, S., Liao, W., Licciardi, G., Simões, M., Tourneret, J.-Y., Veganzones, M., Vivone, G., Wei, Q., Yokoya, N.: Hyperspectral pansharpening: a review. IEEE Geosci. Remote Sens. Mag. 3, 27–46 (2015)CrossRefGoogle Scholar
  14. 14.
    Nascimento, J., Bioucas-Dias, J.: Vertex component analysis: a fast algorithm to unmix hyperspectral data. IEEE Trans. Geosci. Remote Sens. 43, 898–910 (2005)CrossRefGoogle Scholar
  15. 15.
    Romano, Y., Elad, M., Milanfar, P.: The little engine that could: regularization by denoising (RED) arXiv:1611.02862 (2016)
  16. 16.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D: Nonlinear Phenom. 60, 259–268 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Simões, M., Bioucas-Dias, J., Almeida, L., Chanussot, J.: A convex formulation for hyperspectral image superresolution via subspace-based regularization. IEEE Trans. Geosci. Remote Sens. 55, 3373–3388 (2015)CrossRefGoogle Scholar
  18. 18.
    Sreehari, S., Venkatakrishnan, S., Wohlberg, B., Buzzard, G., Drummy, L., Simmons, J., Bouman, C.: Plug-and-play priors for bright field electron tomography and sparse interpolation. IEEE Trans. Comput. Imaging 2(4), 408–423 (2016)MathSciNetGoogle Scholar
  19. 19.
    Teodoro, A., Almeida, M., Figueiredo, M.: Single-frame image denoising and inpainting using Gaussian mixtures. In: ICPRAM, pp. 283–288 (2015)Google Scholar
  20. 20.
    Teodoro, A., Bioucas-Dias, J., Figueiredo, M.: Image restoration and reconstruction using variable splitting and class-adapted image priors. In: IEEE-ICIP (2016)Google Scholar
  21. 21.
    Teodoro, A., Bioucas-Dias, J., Figueiredo, M.: Image restoration with locally selected class-adapted models. In: IEEE-MLSP (2016)Google Scholar
  22. 22.
    Teodoro, A., Bioucas-Dias, J., Figueiredo, M.: Sharpening hyperspectral images using plug-and-play priors. In: Tichavský, P., Babaie-Zadeh, M., Michel, O., Thirion-Moreau, N. (eds.) LVA/ICA 2017. LNCS, vol. 10169, pp. 392–402. Springer, Cham (2017). CrossRefGoogle Scholar
  23. 23.
    Teodoro, A., Bioucas-Dias, J., Figueiredo, M.: Hyperspectral sharpening using scene-adapted Gaussian mixture priors. Preprint arXiv:1702.02445 (2017)
  24. 24.
    Venkatakrishnan, S., Bouman, C., Chu, E., Wohlberg, B.: Plug-and-play priors for model based reconstruction. In: IEEE GlobalSIP, pp. 945–948 (2013)Google Scholar
  25. 25.
    Wei, Q., Bioucas-Dias, J., Dobigeon, N., Tourneret, J.-Y.: Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Trans. Geosci. Remote Sens. 53, 3658–3668 (2015)CrossRefGoogle Scholar
  26. 26.
    Yokoya, N., Grohnfeldt, C., Chanussot, J.: Hyperspectral and multispectral data fusion: a comparative review. IEEE Geosci. Remote Sens. Mag. 5, 29–56 (2017)CrossRefGoogle Scholar
  27. 27.
    Yu, G., Sapiro, G., Mallat, S.: Solving inverse problems with piecewise linear estimators: from Gaussian mixture models to structured sparsity. IEEE Trans. Image Process. 21, 2481–2499 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Zoran, D., Weiss, Y.: From learning models of natural image patches to whole image restoration. In: IEEE-CVPR, pp. 479–486 (2011)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Afonso M. Teodoro
    • 1
  • José M. Bioucas-Dias
    • 1
  • Mário A. T. Figueiredo
    • 1
    Email author
  1. 1.Instituto de Telecomunicações, Instituto Superior TécnicoUniversidade de LisboaLisbonPortugal

Personalised recommendations