Abstract
In recent years, deep learning-based image extrapolation has achieved remarkable improvements. Image extrapolation utilizes the structural and semantic information from the known area of an image to extrapolate the unknown area. In addition, these extrapolative parts not only maintain the consistency of spatial information and structural information with the known area, but also achieve a clear, beautiful, natural and harmonious visual effect. In view of the shortcomings of traditional image extrapolation methods, this paper proposes an image extrapolation method which is based on perceptual loss and style loss. In the paper, we use the perceptual loss and style loss to restrain the generation of the texture and style of images, which improves the distorted and fuzzy structure generated by traditional methods. The perceptual loss and style loss capture the semantic information and the overall style of the known area respectively, which is helpful for the network to grasp the texture and style of images. The experiments on the Places2 and Paris StreetView dataset show that our approach could produce better results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Deepak, P., Philipp, K., Jeff, D., Trevor, D., Alexei, A.E.: Context encoders: feature learning by inpainting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2536–2544 (2016)
Goodfellow, I., Pouget-Abadie, J., Mirza, M.: Generative adversarial nets. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 654–656 (2014)
Coloma, B., Marcelo, B., Vient, C., Guillermo, S., Joan, V.: Filling-in by joint interpolation of vector fields and gray levels. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1200–1211 (2001)
Sun, J., Yuan, L., Jia, J., Shum, H.-Y.: Image completion with structure propagation. ACM Trans. Graph. 24, 861–868 (2005)
Chao, Y., Xin, L., Zhe, L.: High-Resolution image inpainting using multi-scale neural patch synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1457–1460 (2017)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1409–1412 (2014)
Mark, S., Gili, R.: Painting outside the box: image outpainting with GANs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 421–420 (2018)
Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. 36, 1–14 (2017)
Wang, C., Liu, M.Y., Zhu, J.Y., Tao, A., Kautz, J.: High-resolution image synthesis and semantic manipulation with conditional GANs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5–10 (2018)
Chuanxia, Z., Tat-Jen, C., Jianfei, C.: Pluralistic image completion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–5 (2019)
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision – ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417–424 (2000)
Kamyar, N., Eric, B., Tony, J., Faisal, Z., Qureshi, M.: EdgeConnect: generative image inpainting with adversarial edge learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5–7 (2019)
Dolhansky, B., Ferrer, C.C.: Eye in-painting with exemplar generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7902–7911 (2018)
Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1200–1211 (2001)
Hays, J., Efros, A.: Scene completion using millions of photographs. ACM Trans. Graph. 26(3), 4 (2007)
Doersch, C., Singh, S., Gupta, A., Sivic, J., Efros, A.: What makes Paris look like Paris? ACM Trans. Graph. 31(4), 1–9 (2012). https://doi.org/10.1145/2185520.2185597
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: A 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2018)
Acknowledgment
This work was supported by the National Key Research and Development Program of China (No. 2017YFC1502203), and the Sichuan Science and Technology program (2019JDJQ0002, 18MZGC0060, 2018RZ0072, and 2018GZ0184), the major Project of Education Department in Sichuan (17ZA0063).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Ren, Y. et al. (2021). Image Extrapolation Based on Perceptual Loss and Style Loss. In: Wu, X., Wu, K., Wang, C. (eds) Quality, Reliability, Security and Robustness in Heterogeneous Systems. QShine 2020. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 381. Springer, Cham. https://doi.org/10.1007/978-3-030-77569-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-77569-8_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77568-1
Online ISBN: 978-3-030-77569-8
eBook Packages: Computer ScienceComputer Science (R0)