Abstract
Commercial Light-Field cameras provide spatial and angular information, but its limited resolution becomes an important problem in practical use. Different from the conventional images, Light-Field images contain more information of different views that can be used for super-resolution and it makes super-resolution more credible. In this paper, we propose a interpolation based method for Light-Field image super-resolution by taking advantage of the epipolar plane image (EPI) to transfer angular information into spatial information. Firstly, we propose a color recovery framework for undetermined pixels. This framework contains three parts: we estimate the similar-color-diagonal (SCD) for known pixels, we construct a set of filters corresponding to different SCD to generate colors in order to provide a color selection set for undetermined pixel and we propose a W-shaped operator to select a more credible color for undetermined pixel. Finally we use this framework to interpolate EPI and the interpolated EPIs are used to reconstruct a high-resolution image. Experimental results demonstrate that the proposed method outperforms the state-of-art methods for Light-Field spatial super-resolution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adelson, E.H., Bergen, J.R.: The plenoptic function and the elements of early vision, pp. 3–20 (1991)
Bolles, R.C., Baker, H.H., Marimont, D.H.: Epipolar-plane image analysis: an approach to determining structure from motion. Int. J. Comput. Vis. 1(1), 7–55 (1987)
Chen, C., Lin, H., Yu, Z., Kang, S.B., Yu, J.: Light field stereo matching using bilateral statistics of surface cameras (10636919), pp. 1518–1525 (2014)
Cho, D., Lee, M., Kim, S., Tai, Y.W.: Modeling the calibration pipeline of the lytro camera for high quality light-field image reconstruction. In: IEEE International Conference on Computer Vision, pp. 3280–3287 (2014)
Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_13
Honauer, K., Johannsen, O., Kondermann, D., Goldluecke, B.: A dataset and evaluation methodology for depth estimation on 4D light fields. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10113, pp. 19–34. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54187-7_2
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: Computer Vision and Pattern Recognition, pp. 1637–1645 (2016)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Computer Vision and Pattern Recognition, pp. 105–114 (2017)
Mitra, K., Veeraraghavan, A.: Light field denoising, light field superresolution and stereo camera based refocussing using a GMM light field patch prior, pp. 22–28 (2012)
Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, pp. 1874–1883 (2016)
Tao, M.W., Su, J.C., Wang, T.C., Malik, J., Ramamoorthi, R.: Depth estimation and specular removal for glossy surfaces using point and line consistency with light-field cameras. IEEE Trans. Pattern Anal. Mach. Intell. 38(6), 1155–1169 (2016)
Wanner, S., Goldluecke, B.: Spatial and angular variational super-resolution of 4D light fields. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 608–621. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33715-4_44
Wanner, S., Goldluecke, B.: Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 606–619 (2014)
Yoon, Y., Jeon, H., Yoo, D., Lee, J., Kweon, I.S.: Learning a deep convolutional network for light-field image super-resolution, pp. 57–65 (2015)
Zhang, S., Sheng, H., Yang, D., Zhang, J., Xiong, Z.: Micro-lens-based matching for scene recovery in lenslet cameras. IEEE Trans. Image Process. PP(99), 1 (2017). A Publication of the IEEE Signal Processing Society
Zhang, S., Sheng, H., Li, C., Zhang, J., Xiong, Z.: Robust depth estimation for light field via spinning parallelogram operator. Comput. Vis. Image Underst. 145(145), 148–159 (2016)
Zhang, Z., Liu, Y., Dai, Q.: Light field from micro-baseline image pair. In: Computer Vision and Pattern Recognition, pp. 3800–3809 (2015)
Zhou, L.Y., Cai-Xia, S.U., Cao, Y.F.: Image super-resolution via sparse representation. Comput. Eng. Des. (2016)
Acknowledgement
This study is partially supported by the National Key R&D Program of China (No. 2018YFB0505500), the National Natural Science Foundation of China (No. 61635002), the Macao Science and Technology Development Fund (No. 138/2 016/A3), the Program of Introducing Talents of Discipline to Universities and the Open Fund of the State Key Laboratory of Software Development Environment under grant SKLSDE-2017ZX-09, the Project of Experimental Verification of the Basic Commonness and Key Technical Standards of the Industrial Internet network architecture. Thank you for the support from HAWKEYE Group.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Su, B., Sheng, H., Zhang, S., Yang, D., Chen, N., Ke, W. (2018). W-Shaped Selection for Light Field Super-Resolution. In: Liu, W., Giunchiglia, F., Yang, B. (eds) Knowledge Science, Engineering and Management. KSEM 2018. Lecture Notes in Computer Science(), vol 11061. Springer, Cham. https://doi.org/10.1007/978-3-319-99365-2_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-99365-2_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-99364-5
Online ISBN: 978-3-319-99365-2
eBook Packages: Computer ScienceComputer Science (R0)