Abstract
Video streams are ubiquitous in applications such as surveillance, games, and live broadcast. Processing and analyzing these data is challenging because algorithms have to be efficient in order to process the data on the fly. From a theoretical standpoint, video streams have their own specificities – they mix spatial and temporal dimensions, and compared to standard video sequences, half of the information is missing, i.e. the future is unknown. The theoretical part of our work is motivated by the ubiquitous use of the Gaussian kernel in tools such as bilateral filtering and mean-shift segmentation. We formally derive its equivalent for video streams as well as a dedicated expression of isotropic diffusion. Building upon this theoretical ground, we adapt a number of classical algorithms to video streams: bilateral filtering, mean-shift segmentation, and anisotropic diffusion.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Zheng, Y., Lin, S., Kang, S.B.: Single-image vignetting correction. In: Proc. of the conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 461–468. IEEE, Los Alamitos (2006)
Bennett, E.P., McMillan, L.: Video enhancement using per-pixel virtual exposures. ACM Transactions on Graphics 24, 845–852 (2005); Proc. of the ACM SIGGRAPH conf.
Bennett, E.P., Mason, J.L., McMillan, L.: Multispectral bilateral video fusion. IEEE Transactions on Image Processing 16, 1185–1194 (2007)
Liu, C., Freeman, W.T., Szeliski, R., Kang, S.: Noise estimation from a single image. In: Proc. of the Computer Vision and Pattern Recognition Conf. IEEE, Los Alamitos (2006)
Li, Y., Sun, J., Shum, H.Y.: Video object cut and paste. ACM Transactions on Graphics 24, 595–600 (2005); Proc. of the ACM SIGGRAPH conf.
Wang, J., Bhat, P., Colburn, R.A., Agrawala, M., Cohen, M.F.: Video cutout. ACM Transactions on Graphics 24 (2005); Proc. of the ACM SIGGRAPH conf.
Bae, S., Paris, S., Durand, F.: Two-scale tone management for photographic look. ACM Transactions on Graphics 25, 637–645 (2006); Proc. of the ACM SIGGRAPH conf.
DeCarlo, D., Santella, A.: Stylization and abstraction of photographs. In: Proc. of the ACM SIGGRAPH conf. (2002)
Winnemöller, H., Olsen, S.C., Gooch, B.: Real-time video abstraction. ACM Transactions on Graphics 25, 1221–1226 (2006); Proc. of the ACM SIGGRAPH conf.
Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. ACM Transactions on Graphics 26 (2007); Proc. of the ACM SIGGRAPH conf.
Wang, J., Xu, Y., Shum, H.Y., Cohen, M.F.: Video tooning. ACM Transactions on Graphics 23, 294–302 (2004); Proc. of the ACM SIGGRAPH conf.
Wang, J., Thiesson, B., Xu, Y., Cohen, M.F.: Image and video segmentation by anisotropic mean shift. In: Proc. of the European Conf. on Computer Vision (2004)
Bousseau, A., Neyret, F., Thollot, J., Salesin, D.: Video watercolorization using bidirectional texture advection. ACM Transactions on Graphics 26 (2007); Proc. of the ACM SIGGRAPH conf.
Paris, S., Durand, F.: A topological approach to hierarchical segmentation using mean shift. In: Proc. of the IEEE conf. on Computer Vision and Pattern Recognition (2007)
Buades, A., Coll, B., Morel, J.M.: A non local algorithm for image denoising. In: Proc. of the conf. on Computer Vision and Pattern Recognition (2005)
Chen, J., Tang, C.K.: Spatio-temporal Markov random field for video denoising. In: Proc. of the IEEE conf. on Computer Vision and Pattern Recognition (2007)
Drori, I., Leyvand, T., Fleishman, S., Cohen-Or, D., Yeshurun, H.: Video operations in the gradient domain. Technical report, Tel-Aviv University (2004)
Chuang, Y.Y., Agarwala, A., Curless, B., Salesin, D., Szeliski, R.: Video matting of complex scenes. ACM Transactions on Graphics 21 (2002); Proc. of the ACM SIGGRAPH conf.
DeMenthon, D.: Spatio-temporal segmentation of video by hierarchical mean shift analysis. In: Proc. of the Statistical Methods in Video Processing Workshop (2002)
Zitnick, C.L., Jojic, N., Kang, S.B.: Consistent segmentation for optical flow estimation. In: Proc. of the International Conf. on Computer Vision (2005)
Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. In: Proc. of the European Conf. on Computer Vision (2006)
Comaniciu, D., Meer, P.: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis Machine Intelligence 24, 603–619 (2002)
Aubert, G., Kornprobst, P.: Mathematical problems in image processing: Partial Differential Equations and the Calculus of Variations. Applied Mathematical Sciences, vol. 147. Springer, Heidelberg (2002)
Koenderink, J.J.: Scale-time. Biological Cybernetics 58 (1988)
ter Haar Romeny, B.M., Florack, L.M.J., Nielsen, M.: Scale-time kernels and models. In: Proc. of the conf. on Scale-Space and Morphology in Computer Vision (2001)
Lindeberg, T., Fagerström, D.: Scale-space with causal time direction. In: Proc. of European Conf. on Computer Vision (1996)
Lindeberg, T.: Linear spatio-temproal scale-space. In: Proc. of the International Conf. on Scale-Space Teory in Computer Vision (1997)
Klein, A., Sloan, P.P., Finkelstein, A., Cohen, M.F.: Stylized video cubes. In: Proc. of the ACM SIGGRAPH Symposium on Computer Animation (2002)
Kim, J., Woods, J.W.: Spatiotemporal adaptive 3-D Kalman filter for video. IEEE Transactions on Image Processing 6 (1997)
Patti, A.J., Tekalp, A.M., Sezan, M.I.: A new motion-compensated reduced-order model Kalman filter for space-varying restoration of progressive and interlaced video. IEEE Transactions on Image Processing 7 (1998)
Bennett, E.P., McMillan, L.: Computational time-lapse video. ACM Transactions on Graphics 26 (2007); Proc. of the ACM SIGGRAPH conf.
Koenderink, J.J.: The structure of images. Biological Cybernetics 50 (1984)
Guo, H.Y., Li, Y.Q., Wu, K.: Difference discrete variational principle, Euler-Lagrange cohomology and symplectic, multisymplectic structures. ArXiv Math. Physics e-prints (2001)
Deriche, R.: Recursively implementating the Gaussian and its derivatives. Technical Report RR-1893, INRIA (1993)
Aurich, V., Weule, J.: Non-linear gaussian filters performing edge preserving diffusion. In: Proc. of the DAGM Symposium (1995)
Smith, S.M., Brady, J.M.: SUSAN – a new approach to low level image processing. International Journal of Computer Vision 23, 45–78 (1997)
Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proc. of the International Conf. on Computer Vision, pp. 839–846. IEEE, Los Alamitos (1998)
Paris, S., Kornprobst, P., Tumblin, J., Durand, F.: A gentle introduction to bilateral filtering and its applications. In: Course at the ACM SIGGRAPH conf. (2007)
Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Transactions Pattern Analysis Machine Intelligence 12, 629–639 (1990)
Horn, B.K.P.: Determining lightness from an image. Computer Graphics and Image Processing 3 (1974)
Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Transactions on Graphics 22 (2003); Proc. of the ACM SIGGRAPH conf.
Levin, A., Zomet, A., Peleg, S., Weiss, Y.: Seamless image stitching in the gradient domain. In: Proc. of the European Conf. on Computer Vision (2006)
Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. In: Proc. of the International Conf. on Computer Vision (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Electronic Supplementary Material
Supplementary material(26,420 KB)
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Paris, S. (2008). Edge-Preserving Smoothing and Mean-Shift Segmentation of Video Streams. In: Forsyth, D., Torr, P., Zisserman, A. (eds) Computer Vision – ECCV 2008. ECCV 2008. Lecture Notes in Computer Science, vol 5303. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88688-4_34
Download citation
DOI: https://doi.org/10.1007/978-3-540-88688-4_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88685-3
Online ISBN: 978-3-540-88688-4
eBook Packages: Computer ScienceComputer Science (R0)