Skip to main content
SpringerLink
Log in

Intelligent Texture Reconstruction of Missing Data in Video Sequences Using Neural Networks

  • Conference paper
Advanced Techniques for Knowledge Engineering and Innovative Applications

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 246))

Abstract

The paper provides an intelligent method of texture reconstruction after removal of non-disabled objects or artifacts in video sequences. Data under subtitles, logotypes, damages of information medium or small size objects are referred to as missing data. A novel implementation of separated neural network was used to receive spatial texture estimations in missing data region. Usually several types of textures are located under removed object. A fast wave algorithm was developed for boundary interpolation between different types of texture into a missing data region. Three strategies of wave algorithm for contour optimization were suggested. A fully connected one-level neural network was applied for choice of texture inpainting method (blurring, texture tile, and texture synthesis). The proposed technique was tested for visual reconstruction of missing text regions (subtitles, logotypes) and missing objects with area less 8-12% of frame in animation and movies. In the first case, a simplified decision without stage of boundaries approximation may be applied; in the second case, the reconstruction results are significantly determined by a background complexity and motion in scene.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Haralick, R.M.: Statistical and structural approaches to texture. Proceedings of the IEEE 67(5), 786–804 (1979)

    Article  Google Scholar 

  2. Funakubo, N.: Region segmentation of biomedical tissue image using color texture features. In: Proceedings of the 7th Int. Conf. on Pattern Recognition, vol. 1, pp. 30–32 (1984)

    Google Scholar 

  3. Song-Sheng, L., Jernigan, M.E.: Texture Analysis and Discrimination in Additive Noise. Comput. Vision, Graph., Image Process. 49(1), 52–67 (1990)

    Article  Google Scholar 

  4. Ojeda, S., Vallejos, R., Bustos, O.: A new image segmentation algorithm with applications to image inpainting. Computational Statistics and Data Analysis 54, 2082–2093 (2010)

    Article  MathSciNet  Google Scholar 

  5. Tsai, J.-J., Chen, N.-J., Fang, W.-C., Chen, J.-S.: A fast image reconstruction algorithm for continuous wave diffuse optical tomography. In: IEEE/NIH Life Science Systems and Ap-plications Workshop (LiSSA), pp. 92–95 (2011)

    Google Scholar 

  6. Quiney, H.M., Nugent, K.A., Peele, A.G.: Iterative image reconstruction algorithms using wave-front intensity and phase variation. Optics Letters 30(13), 1638–1640 (2005)

    Article  Google Scholar 

  7. Cho, D., Bui, T.D.: Image inpainting using wavelet-based inter- and intra-scale dependency. In: International Conference on Pattern Recognition (ICPR), Tampa, FL, pp. 1–4 (2008)

    Google Scholar 

  8. Padmavathi, S., Priyalakshmi, B., Soman, K.P.: Hierarchical Digital Image Inpainting Using Wavelets. Signal & Image Processing: An International Journal (SIPIJ) 3(4), 85–93 (2012)

    Google Scholar 

  9. Du, X., Cho, D., Bui, T.D.: Image segmentation and inpainting using hierarchical level set and texture mapping. Signal Processing 91(4), 852–863 (2011)

    Article  MATH  Google Scholar 

  10. Ghoniem, M., Chahir, Y., Elmoataz, A.: Nonlocal video denoising, simplification and inpainting using discrete regularization on graphs. Signal Processing 90, 2445–2455 (2010)

    Article  MATH  Google Scholar 

  11. Lefebvre, A., Corpetti, T., Moy, L.H.: Estimation of the orientation of textured patterns via wavelet analysis. Pattern Recognition Letters 32, 190–196 (2011)

    Article  Google Scholar 

  12. Arias, P., Caselles, V., Sapiro, G.: A variational framework for non-local image inpainting. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 345–358. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  13. Bugeau, A., Bertalmio, M., Caselles, V., Sapiro, G.: A comprehensive framework for image inpainting. IEEE Transactions on Image Processing 19, 2634–2645 (2010)

    Article  MathSciNet  Google Scholar 

  14. Hedjam, R., Mignotte, M.: A hierarchical graph-based Markovian clustering approach for the unsupervised segmentation of textured color images. In: Proceedings of the International Conference on Image Processing (ICIP 2009), pp. 1365–1368 (2009)

    Google Scholar 

  15. Krinidis, M.: Pitas, placeI.: Color texture segmentation based on the modal energy of deformable surfaces. IEEE Transactions on Image Processing 18(7), 1613–1622 (2009)

    Article  MathSciNet  Google Scholar 

  16. Nammalwar, P., Ghita, O., Whelan, P.F.: A generic framework for color texture segmentation. Sensor Review 30(1), 69–79 (2010)

    Article  Google Scholar 

  17. Ilea, D.E., Whelan, P.F.: Image segmentation based on the integration of color–texture descriptors – A review. Pattern Recognition 44, 2479–2501 (2011)

    Article  MATH  Google Scholar 

  18. Pietikainen, M., Maenpaa, T., Viertola, J.: Color texture classification with color histograms and local binary patterns. In: Proceedings of the Second Int. Workshop on Texture Analysis and Synthesis, Copenhagen, Denmark, pp. 109–112 (2006)

    Google Scholar 

  19. Chen, K.M., Chen, S.Y.: Color texture segmentation using feature distributions. Pattern Recognition Letters 23(7), 755–771 (2002)

    Article  MATH  Google Scholar 

  20. Garcia Ugarriza, L., Saber, E., Vantaram, S.R., Amuso, V., Shaw, M., Bhaskar, R.: Automatic image segmentation by dynamic region growth and multi- resolution merging. IEEE Transactions on Image Processing 18(10), 2275–2288 (2009)

    Article  MathSciNet  Google Scholar 

  21. Fadili, M., Starck, J.L., Murtagh, F.: Inpainting and zooming using sparse representations. Computer Journal 52, 64–79 (2009)

    Article  Google Scholar 

  22. Xu, Z., Jian, S.: Image inpainting by patch propagation using patch sparsity. IEEE Transactions on Image Processing 19, 1153–1165 (2010)

    Article  MathSciNet  Google Scholar 

  23. Lowe, D.: Distinctive Image Features from Scale-invariant Keypoints. Int. J. of Computer Vision 60, 91–110 (2004)

    Article  Google Scholar 

  24. Favorskaya, M., Zotin, A., Damov, M.: Intelligent Inpainting System for Texture Reconstruction in Videos with Text Removal. In: Proceedings of Int. Congress on Ultra Modern Telecommunications and Control Systems, ICUMT 2010, pp. 867–874 (2010)

    Google Scholar 

  25. Vacha, P., Haindl, M., Suk, T.: Colour and rotation invariant textural features based on Markov random fields. Pattern Recognition Letters 32, 771–779 (2011)

    Article  Google Scholar 

  26. Khan, J.F., Adhami, R.R., Bhuiyan, S.M.A.: A customized Gabor filter for unsupervised colour image segmentation. Image and Vision Computing 27(4), 489–501 (2009)

    Article  Google Scholar 

  27. Al-Takrouri, S., Savkin, A.V.: A model validation approach to texture recognition and inpainting. Pattern Recognition 43, 2054–2067 (2010)

    Article  MATH  Google Scholar 

  28. Favorskaya, M.N., Petukhov, N.Y.: Recognition of natural objects on air photographs using neural networks. Optoelectronics, Instrumentation and Data Processing 47(3), 233–238 (2011)

    Article  Google Scholar 

  29. Anupam Goyal, P., Diwakar, S.: Fast and Enhanced Algorithm for Exemplar Based Image Inpainting. In: 4th Pacific-Rim Symposium on Image and Video Technology (PSIVT), pp. 325–330 (2010)

    Google Scholar 

  30. Guo, H., An, J.: Image Restoration with Morphological Erosion and Exemplar-Based Texture Synthesis. In: 6th International Conference on Wireless Communications Networking and Mobile Computing (WiCOM), pp. 1–4 (2010)

    Google Scholar 

  31. Shalini, S.S., Menaka, D.: Exemplar Based Image and Video Inpainting. International Journal of Communications and Engineering 04(4), 112–119 (2012)

    Google Scholar 

  32. Zhang, Q., Lin, J.: Exemplar-Based Image Inpainting Using Color Distribution Analysis. Journal of Information Science and Engineering 28, 641–654 (2012)

    Google Scholar 

  33. Huan, X., Murali, B., Ali, A.L.: Image restoration based on the fast marching method and block based sampling. Computer Vision and Image Understanding 114, 847–856 (2010)

    Article  Google Scholar 

  34. Boulanger, J., Kervrann, C., Bouthemy, P.: Space–time adaptation for patch-based image sequence restoration. IEEE Transactions on PAMI 29, 1096–1102 (2007)

    Article  Google Scholar 

  35. Protter, M., Elad, M., Takeda, H., Milanfar, P.: Generalizing the non- local-means to super-resolution reconstruction. IEEE Transactions on Image Processing 18(1) (2009)

    Google Scholar 

  36. Harris, C., Stephens, M.: A combined corner and edge detection. In: Proceedings of The Fourth Alvey Vision Conference, Manchester, UK, pp. 147–151 (1988)

    Google Scholar 

  37. Lucas, B.D., Kanade, T.: An Iterative Image Registration Technique with an Application to Stereo Vision. In: International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Siberian State Aerospace University, Krasnoyarsk, 660014, Russian Federation

    Margarita Favorskaya, Mikhail Damov & Alexander Zotin

Authors
  1. Margarita Favorskaya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mikhail Damov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Zotin
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Electrical and Information Engineering, University of South Australia, Mawson Lake Campus, 5095, Adelaide, SA, Australia

    Jeffrey W. Tweedale

  2. School of Electrical and Information Engineering, University of South Australia, Mawson Lakes Campus, 5095, Adelaide, SA, Australia

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Favorskaya, M., Damov, M., Zotin, A. (2013). Intelligent Texture Reconstruction of Missing Data in Video Sequences Using Neural Networks. In: Tweedale, J.W., Jain, L.C. (eds) Advanced Techniques for Knowledge Engineering and Innovative Applications. Communications in Computer and Information Science, vol 246. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-42017-7_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-42017-7_12

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-42016-0

  • Online ISBN: 978-3-642-42017-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation