Abstract
Video decaptioning is the task of removing overlayed text in the video sequences and filling it with coherent and semantically meaningful content. In this work, we propose a generative CNN to do caption detection and decaptioning task jointly in an end-to-end fashion. Our network does frame level decaptioning and thus the same network can be adapted to do image decaptioning or other similar tasks. Our network is capable of inpainting video frames that contain text overlays in various size, background, color, and location. All the experiments were performed on the dataset provided in the ECCV’18 Satellite Workshop Chalearn LAP Inpainting Competition Track 2 - Video decaptioning. We also secured the third rank in the competition. Most of our work is inspired by previous work on image generation and image inpainting.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in neural information processing systems, pp. 2672–2680 (2014)
Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Transactions on Graphics (TOG) 36(4), 107 (2017)
Jia, Y.T., Hu, S.M., Martin, R.R.: Video completion using tracking and fragment merging. The Visual Computer 21(8–10), 601–610 (2005)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision, pp. 694–711. Springer (2016)
Newson, A., Almansa, A., Fradet, M., Gousseau, Y., Pérez, P.: Video inpainting of complex scenes. SIAM Journal on Imaging Sciences 7(4), 1993–2019 (2014)
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: Feature learning by inpainting. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2536–2544 (2016)
Shen, Y., Lu, F., Cao, X., Foroosh, H.: Video completion for perspective camera under constrained motion. In: null, pp. 63–66. IEEE (2006)
Wexler, Y., Shechtman, E., Irani, M.: Space-time video completion. In: null, pp. 120–127. IEEE (2004)
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Generative image inpainting with contextual attention. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5505–5514 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Patel, V., Pandey, A. (2019). Joint Caption Detection and Inpainting Using Generative Network. In: Escalera, S., Ayache, S., Wan, J., Madadi, M., Güçlü, U., Baró, X. (eds) Inpainting and Denoising Challenges. The Springer Series on Challenges in Machine Learning. Springer, Cham. https://doi.org/10.1007/978-3-030-25614-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-25614-2_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-25613-5
Online ISBN: 978-3-030-25614-2
eBook Packages: Computer ScienceComputer Science (R0)