Advertisement

W-Shaped Selection for Light Field Super-Resolution

  • Bing Su
  • Hao Sheng
  • Shuo Zhang
  • Da Yang
  • Nengcheng Chen
  • Wei Ke
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11061)

Abstract

Commercial Light-Field cameras provide spatial and angular information, but its limited resolution becomes an important problem in practical use. Different from the conventional images, Light-Field images contain more information of different views that can be used for super-resolution and it makes super-resolution more credible. In this paper, we propose a interpolation based method for Light-Field image super-resolution by taking advantage of the epipolar plane image (EPI) to transfer angular information into spatial information. Firstly, we propose a color recovery framework for undetermined pixels. This framework contains three parts: we estimate the similar-color-diagonal (SCD) for known pixels, we construct a set of filters corresponding to different SCD to generate colors in order to provide a color selection set for undetermined pixel and we propose a W-shaped operator to select a more credible color for undetermined pixel. Finally we use this framework to interpolate EPI and the interpolated EPIs are used to reconstruct a high-resolution image. Experimental results demonstrate that the proposed method outperforms the state-of-art methods for Light-Field spatial super-resolution.

Keywords

Light-field Super-resolution Interpolation W-shaped operator 

Notes

Acknowledgement

This study is partially supported by the National Key R&D Program of China (No. 2018YFB0505500), the National Natural Science Foundation of China (No. 61635002), the Macao Science and Technology Development Fund (No. 138/2 016/A3), the Program of Introducing Talents of Discipline to Universities and the Open Fund of the State Key Laboratory of Software Development Environment under grant SKLSDE-2017ZX-09, the Project of Experimental Verification of the Basic Commonness and Key Technical Standards of the Industrial Internet network architecture. Thank you for the support from HAWKEYE Group.

References

  1. 1.
    Adelson, E.H., Bergen, J.R.: The plenoptic function and the elements of early vision, pp. 3–20 (1991)Google Scholar
  2. 2.
    Bolles, R.C., Baker, H.H., Marimont, D.H.: Epipolar-plane image analysis: an approach to determining structure from motion. Int. J. Comput. Vis. 1(1), 7–55 (1987)CrossRefGoogle Scholar
  3. 3.
    Chen, C., Lin, H., Yu, Z., Kang, S.B., Yu, J.: Light field stereo matching using bilateral statistics of surface cameras (10636919), pp. 1518–1525 (2014)Google Scholar
  4. 4.
    Cho, D., Lee, M., Kim, S., Tai, Y.W.: Modeling the calibration pipeline of the lytro camera for high quality light-field image reconstruction. In: IEEE International Conference on Computer Vision, pp. 3280–3287 (2014)Google Scholar
  5. 5.
    Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10593-2_13CrossRefGoogle Scholar
  6. 6.
    Honauer, K., Johannsen, O., Kondermann, D., Goldluecke, B.: A dataset and evaluation methodology for depth estimation on 4D light fields. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10113, pp. 19–34. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-54187-7_2CrossRefGoogle Scholar
  7. 7.
    Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46475-6_43CrossRefGoogle Scholar
  8. 8.
    Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: Computer Vision and Pattern Recognition, pp. 1637–1645 (2016)Google Scholar
  9. 9.
    Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Computer Vision and Pattern Recognition, pp. 105–114 (2017)Google Scholar
  10. 10.
    Mitra, K., Veeraraghavan, A.: Light field denoising, light field superresolution and stereo camera based refocussing using a GMM light field patch prior, pp. 22–28 (2012)Google Scholar
  11. 11.
    Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, pp. 1874–1883 (2016)Google Scholar
  12. 12.
    Tao, M.W., Su, J.C., Wang, T.C., Malik, J., Ramamoorthi, R.: Depth estimation and specular removal for glossy surfaces using point and line consistency with light-field cameras. IEEE Trans. Pattern Anal. Mach. Intell. 38(6), 1155–1169 (2016)CrossRefGoogle Scholar
  13. 13.
    Wanner, S., Goldluecke, B.: Spatial and angular variational super-resolution of 4D light fields. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 608–621. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33715-4_44CrossRefGoogle Scholar
  14. 14.
    Wanner, S., Goldluecke, B.: Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 606–619 (2014)CrossRefGoogle Scholar
  15. 15.
    Yoon, Y., Jeon, H., Yoo, D., Lee, J., Kweon, I.S.: Learning a deep convolutional network for light-field image super-resolution, pp. 57–65 (2015)Google Scholar
  16. 16.
    Zhang, S., Sheng, H., Yang, D., Zhang, J., Xiong, Z.: Micro-lens-based matching for scene recovery in lenslet cameras. IEEE Trans. Image Process. PP(99), 1 (2017). A Publication of the IEEE Signal Processing SocietyGoogle Scholar
  17. 17.
    Zhang, S., Sheng, H., Li, C., Zhang, J., Xiong, Z.: Robust depth estimation for light field via spinning parallelogram operator. Comput. Vis. Image Underst. 145(145), 148–159 (2016)CrossRefGoogle Scholar
  18. 18.
    Zhang, Z., Liu, Y., Dai, Q.: Light field from micro-baseline image pair. In: Computer Vision and Pattern Recognition, pp. 3800–3809 (2015)Google Scholar
  19. 19.
    Zhou, L.Y., Cai-Xia, S.U., Cao, Y.F.: Image super-resolution via sparse representation. Comput. Eng. Des. (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.State Key Laboratory of Software Development Environment, School of Computer Science and EngineeringBeihang UniversityBeijingChina
  2. 2.State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS)Wuhan UniversityWuhanPeople’s Republic of China
  3. 3.Macao Polytechnic InstituteMacaoPeople’s Republic of China

Personalised recommendations