Sensing and Imaging

, 20:14 | Cite as

Extended Super Resolution of Hyperspectral Images via Non-negative Sparse Coding

  • Maneesh PawarEmail author
  • K. S. Venkatesh
Original Paper


Often high resolution (HR) RGB images generated by sparse sampling of the visible spectrum fail to produce differentiable modality for computer vision tasks, Hence computer vision tasks have to rely on gross structures in an image like corners, edges etc. instead of just the recorded reflectance of objects or materials at each pixel in a scene. In contrast to RGB, hyperspectral imaging allows pixels to record reflectance of the scene over multiple contiguous bands, which results in rich differentiable modalities. However, hyperspectral imaging, despite having a growing number of applications from agriculture, surveillance, mineralogy, food processing to eye care, is hitherto restricted to low spatial resolution imaging due to sensor hardware limitations. In this paper, we propose a hyperspectral super resolution technique to produce a high resolution (HR) hyperspectral image with a spectral support of 400–1020 nm from a low resolution (LR) hyperspectral image of the same spectral support and a high resolution multispectral (RGB) image with reduced spectral support of 400–700 nm. In the first step, we generate a HR hyperspectral prior by estimating HR hyperspectral band images in the spectral support of 400–700 nm by detail transfer and alternating iterative minimization. In the next step, we use the generated HR prior to further estimate the HR hyperspectral images for 710–1020 nm bands by learning a non-negative dictionary of reflectance spectra signatures of all the materials present in the scene from the LR hyperspectral image with spectral support in 400–1020 nm. With the estimated HR hyperspectral prior and learned dictionary, we predict the non-negative sparse codes for HR hyperspectral band images in the band of 710–1020 nm.


Hyperspectral imaging Non-negative matrix factorization Alternative direction multiplier method Non-negative sparse coding 



This research work is a part of postgraduate thesis dissertation, financially supported by employer of first author which is Instrument Research and Development Establishment, Dehradun Uttarakhand, India.


  1. 1.
    Aharon, M., Elad, M., & Bruckstein, A. (2006). K-svd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on signal processing, 54(11), 4311–4322.zbMATHGoogle Scholar
  2. 2.
    Aiazzi, B., Alparone, L., Baronti, S., & Garzelli, A. (2002). Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Transactions on Geoscience and Remote Sensing, 40(10), 2300–2312.Google Scholar
  3. 3.
    Akhtar, N., Shafait, F., & Mian, A. (2015). Bayesian sparse representation for hyperspectral image super resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3631–3640).Google Scholar
  4. 4.
    Alparone, L., Wald, L., Chanussot, J., Thomas, C., Gamba, P., & Bruce, L. M. (2007). Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data-fusion contest. IEEE Transactions on Geoscience and Remote Sensing, 45(10), 3012–3021.Google Scholar
  5. 5.
    Arad, B., & Ben-Shahar, O. (2016). Sparse recovery of hyperspectral signal from natural rgb images. In European conference on computer vision (pp. 19–34). Springer.Google Scholar
  6. 6.
    Bioucas-Dias, J. M., & Figueiredo, M. A. T. (2010). Alternating direction algorithms for constrained sparse regression: Application to hyperspectral unmixing. In 2010 2nd workshop on hyperspectral image and signal processing: Evolution in remote sensing (WHISPERS) (pp. 1–4). IEEE.Google Scholar
  7. 7.
    Bolte, J., Sabach, S., & Teboulle, M. (2014). Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Mathematical Programming, 146(1–2), 459–494.MathSciNetzbMATHGoogle Scholar
  8. 8.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine Learning, 3(1), 1–122.zbMATHGoogle Scholar
  9. 9.
    Burt, P., & Adelson, E. (1983). The laplacian pyramid as a compact image code. IEEE Transactions on communications, 31(4), 532–540.Google Scholar
  10. 10.
    Carper, W. J. (1990). The use of intensity-hue-saturation transformations for merging spot panchromatic and multispectral image data. Photogrammetric Engineering and Remote Sensing, 56(4), 457–467.Google Scholar
  11. 11.
    Chavez, P., Sides, S. C., Anderson, J. A., et al. (1991). Comparison of three different methods to merge multiresolution and multispectral data- landsat tm and spot panchromatic. Photogrammetric Engineering and Remote Sensing, 57(3), 295–303.Google Scholar
  12. 12.
    Dong, W., Fazuo, F., Shi, G., Cao, X., Jinjian, W., Li, G., et al. (2016). Hyperspectral image super-resolution via non-negative structured sparse representation. IEEE Transactions on Image Processing, 25(5), 2337–2352.MathSciNetzbMATHGoogle Scholar
  13. 13.
    Du, Q., Younan, N. H., King, R., & Shah, V. P. (2007). On the performance evaluation of pan-sharpening techniques. IEEE Geoscience and Remote Sensing Letters, 4(4), 518–522.Google Scholar
  14. 14.
    Elad, M. (2010). Pursuit algorithms. Sparse and redundant representations from theory to applications in signal and image processing (pp. 55–77). New York: Springer.zbMATHGoogle Scholar
  15. 15.
    Gonzalez, E. F., & Zhang, Y. (2005). Accelerating the Lee-Seung algorithm for nonnegative matrix factorization. Technical report.Google Scholar
  16. 16.
    Hu, J., & Li, S. (2011). Fusion of panchromatic and multispectral images using multiscale dual bilateral filter. In 2011 18th IEEE international conference on image processing (ICIP) (pp. 1489–1492). IEEE.Google Scholar
  17. 17.
    Hu, J., & Li, S. (2012). The multiscale directional bilateral filter and its application to multisensor image fusion. Information Fusion, 13(3), 196–206.Google Scholar
  18. 18.
    Imai, F. H., & Berns, R. S. (1998). High-resolution multi-spectral image archives: a hybrid approach. In Color and imaging conference (Vol. 1998, pp. 224–227). Society for Imaging Science and Technology.Google Scholar
  19. 19.
    Kawakami, R., Matsushita, Y., Wright, J., Ben-Ezra, M., Tai, Y.-W., & Ikeuchi, K. (2011). High-resolution hyperspectral imaging via matrix factorization. In 2011 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 2329–2336). IEEE.Google Scholar
  20. 20.
    Keshava, N., & Mustard, J. F. (2002). Spectral unmixing. IEEE Signal Processing Magazine, 19(1), 44–57.Google Scholar
  21. 21.
    Lanaras, C., Baltsavias, E., & Schindler, K. (2015). Hyperspectral super-resolution by coupled spectral unmixing. In Proceedings of the IEEE international conference on computer vision (pp. 3586–3594).Google Scholar
  22. 22.
    Lee, D. D., & Seung, H. S. (2001). Algorithms for non-negative matrix factorization. In Advances in neural information processing systems (pp. 556–562).Google Scholar
  23. 23.
    Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2009). Online dictionary learning for sparse coding. In Proceedings of the 26th annual international conference on machine learning (pp. 689–696). ACM.Google Scholar
  24. 24.
    Manolakis, D., Marden, D., & Shaw, G. A. (2003). Hyperspectral image processing for automatic target detection applications. Lincoln Laboratory Journal, 14(1), 79–116.Google Scholar
  25. 25.
    Nascimento, J. M. P., & Dias, J. M. B. (2005). Vertex component analysis: A fast algorithm to unmix hyperspectral data. IEEE Transactions on Geoscience and Remote Sensing, 43(4), 898–910.Google Scholar
  26. 26.
    Paatero, P., & Tapper, U. (1994). Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics, 5(2), 111–126.Google Scholar
  27. 27.
    Pan, Z., Healey, G., Prasad, M., & Tromberg, B. (2003). Face recognition in hyperspectral images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12), 1552–1560.Google Scholar
  28. 28.
    Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., & Toyama, K. (2004). Digital photography with flash and no-flash image pairs. ACM Transactions on Graphics (TOG), 23(3), 664–672.Google Scholar
  29. 29.
    Resmini, R. G., Kappus, M. E., Aldrich, W. S., Harsanyi, J. C., & Anderson, M. (1997). Mineral mapping with hyperspectral digital imagery collection experiment (hydice) sensor data at cuprite, Nevada, USA. International Journal of Remote Sensing, 18(7), 1553–1570.Google Scholar
  30. 30.
    Sun, D. L., & Fevotte, C. (2014). Alternating direction method of multipliers for non-negative matrix factorization with the beta-divergence. In 2014 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 6201–6205). IEEE.Google Scholar
  31. 31.
    Van Nguyen, H., Banerjee, A., & Chellappa, R. (2010). Tracking via object reflectance using a hyperspectral video camera. In 2010 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW) (pp. 44–51). IEEE.Google Scholar
  32. 32.
    Wald, L. (2000). Quality of high resolution synthesised images: Is there a simple criterion? In Third conference “Fusion of Earth data: Merging point measurements, raster maps and remotely sensed images” (pp. 99–103). SEE/URISCA.Google Scholar
  33. 33.
    Wang, Z., Ziou, D., Armenakis, C., Li, D., & Li, Q. (2005). A comparative analysis of image fusion methods. IEEE Transactions on Geoscience and Remote Sensing, 43(6), 1391–1402.Google Scholar
  34. 34.
    Wei, Q., Bioucas-Dias, J., Dobigeon, N., & Tourneret, J.-Y. (2015). Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 53(7), 3658–3668.Google Scholar
  35. 35.
    Wycoff, E., Chan, T.-H., Jia, K., Ma, W.-K., & Ma, Y. (2013). A non-negative sparse promoting algorithm for high resolution hyperspectral imaging. In 2013 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 1409–1413). IEEE.Google Scholar
  36. 36.
    Yasuma, F., Mitsunaga, T., Iso, D., & Nayar, S. K. (2010). Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum. IEEE Transactions on Image Processing, 19(9), 2241–2253.MathSciNetzbMATHGoogle Scholar
  37. 37.
    Yokoya, N., Yairi, T., & Iwasaki, A. (2012). Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion. IEEE Transactions on Geoscience and Remote Sensing, 50(2), 528–537.Google Scholar
  38. 38.
    Yuhas, R. H., Goetz, A. F. H., & Boardman, J. W. (1992). Discrimination among semi-arid landscape endmembers using the spectral angle mapper (sam) algorithm. JPL, Summaries of the third annual JPL airborne geoscience workshop (Vol. 1, pp. 147–149).Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.IIT Kanpur IndiaKanpurIndia

Personalised recommendations