Abstract
The ability to generate intermediate frames between two given images in a video sequence is an essential task for video restoration and video post-processing. In addition, restoration requires robust denoising algorithms, must handle corrupted frames and recover from impaired frames accordingly. In this paper we present a unified framework for all these tasks. In our approach we use a variant of the TV-L1 denoising algorithm that operates on image sequences in a space-time volume. The temporal derivative is modified to take the pixels’ movement into account. In order to steer the temporal gradient in the desired direction we utilize optical flow to estimate the velocity vectors between consecutive frames. We demonstrate our approach on impaired movie sequences as well as on benchmark datasets where the ground-truth is known.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kokaram, A.C.: On Missing Data Treatment for Degraded Video and Film Archives: A Survey and a New Bayesian Approach. IEEE Trans. on IP (2004)
Chen, S.E., Williams, L.: View Interpolation for Image Synthesis. In: SIGGRAPH (1993)
McMillan, L., Bishop, G.: Plenoptic Modeling: An Image-Based Rendering System. In: ACM SIGGRAPH (1995)
Faugeras, O., Robert, L.: What Can Two Images Tell Us About a Third One? International Journal of Computer Vision (1996)
Wexler, Y., Sashua, A.: On the Synthesis of Dynamic Scenes from Reference Views. In: IEEE Conference on Computer Vision and Pattern Recognition (2000)
Vedula, S., Baker, S., Kanade, T.: Image-based spatio-temporal modeling and view interpolation of dynamic events. ACM Transactions on Graphics, TOG (2005)
Beier, T., Neely, S.: Feature-Based Image Metamorphosis. In: ACM SIGGRAPH (1992)
Seitz, S.M., Dyer, C.R.: View Morphing. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (1996)
Wolberg, G.: Image morphing: a survey. The Visual Computer (1998)
Schaefer, S., McPhail, T., Warren, J.: Image deformation using moving least squares. ACM Transactions on Graphics, TOG (2006)
Stich, T., Linz, C., Albuquerque, G., Magnor, M.: View and Time Interpolation in Image Space. Computer Graphics Forum (2008)
Mahajan, D., Huang, F.C., Matusik, W., Ramamoorthi, R., Belhumeur, P.: Moving Gradients: A Path-Based Method for Plausible Image Interpolation. In: ACM SIGGRAPH (2009)
Horn, B.K., Schunck, B.G.: Determining Optical Flow. Artificial Intelligence (1981)
Borzì, A., Ito, K., Kunisch, K.: Optimal Control Formulation for Determining Optical Flow. SIAM Journal on Scientific Computing (2006)
Chen, K., Lorenz, D.A.: Image Sequence Interpolation Using Optimal Control. Journal of Mathematical Imaging and Vision (2011)
Rudin, L.I., Osher, S.J., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena (1992)
Nikolova, M.: A Variational Approach to Remove Outliers and Impulse Noise. Journal of Mathematical Imaging and Vision (2004)
Chan, T.F., Esedoglu, S.: Aspects of Total Variation Regularized L[sup 1] Function Approximation. SIAM Journal of Applied Mathematics (2005)
Pock, T.: Fast Total Variation for Computer Vision. PhD thesis, Graz University of Technology (2008)
Mansouri, A.R., Mitiche, A., Aron, M.: PDE-based region tracking without motion computation by joint space-time segmentation. In: ICIP (2003)
Unger, M., Mauthner, T., Pock, T., Bischof, H.: Tracking as Segmentation of Spatial-Temporal Volumes by Anisotropic Weighted TV. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 193–206. Springer, Heidelberg (2009)
Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H.: Anisotropic Huber-L1 Optical Flow. In: BMVC (2009)
Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A Database and Evaluation Methodology for Optical Flow. IJCV (2011)
Barron, J.L., Fleet, D., Beaucheming, S.S.: Performance of optical flow techniques. International Journal of Computer Vision (1994)
Beauchemin, S.S., Barron, J.L.: The Computation of Optical Flow. ACM Computing Surveys (1995)
Aubert, G., Deriche, R., Kornprobst, P.: Computing Optical Flow via Variational Techniques. SIAM J. Appl. Math. (2000)
Fleet, D.J., Weiss, Y.: Optical Flow Estimation (2006)
Chambolle, A., Pock, T.: A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging. J. of Math. Imaging and Vision (2010)
Berkels, B., Kondermann, C., Garbe, C., Rumpf, M.: Reconstructing Optical Flow Fields by Motion Inpainting. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 388–400. Springer, Heidelberg (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Werlberger, M., Pock, T., Unger, M., Bischof, H. (2011). Optical Flow Guided TV-L1 Video Interpolation and Restoration. In: Boykov, Y., Kahl, F., Lempitsky, V., Schmidt, F.R. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2011. Lecture Notes in Computer Science, vol 6819. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23094-3_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-23094-3_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23093-6
Online ISBN: 978-3-642-23094-3
eBook Packages: Computer ScienceComputer Science (R0)