Abstract
The goal of this work is to build video cameras whose spatial and temporal resolutions can be changed post-capture depending on the scene. Building such cameras is difficult due to two reasons. First, current video cameras allow the same spatial resolution and frame rate for the entire captured spatio-temporal volume. Second, both these parameters are fixed before the scene is captured. We propose different components of video camera design: a sampling scheme, processing of captured data and hardware that offer post-capture variable spatial and temporal resolutions, independently at each image location. Using the motion information in the captured data, the correct resolution for each location is decided automatically. Our techniques make it possible to capture fast moving objects without motion blur, while simultaneously preserving high-spatial resolution for static scene parts within the same video sequence. Our sampling scheme requires a fast per-pixel shutter on the sensor-array, which we have implemented using a co-located camera-projector system.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Zabrodsky, H., Peleg, S.: Attentive transmission. J. of Visual Comm. and Image Representation 1 (1990)
Baraniuk, R.: Compressive sensing. IEEE Signal Processing Magazine 24 (2007)
Peers, P., Mahajan, D.K., Lamond, B., Ghosh, A., Matusik, W., Ramamoorthi, R., Debevec, P.: Compressive light transport sensing. ACM Trans. Graph 28 (2009)
Nayar, S.K., Mitsunaga, T.: High dynamic range imaging: spatially varying pixel exposures. In: IEEE CVPR (2000)
Narasimhan, S.G., Nayar, S.K.: Enhancing resolution along multiple imaging dimensions using assorted pixels. PAMI 27 (2005)
Ng, R.: Fourier slice photography. ACM Trans. Graphics 24, 735–744 (2005)
Horstmeyer, R., Euliss, G., Athale, R., Levoy, M.: Flexible multimodal camera using a light field architecture. In: ICCP (2009)
Ben-Ezra, M., Zomet, A., Nayar, S.K.: Video super-resolution using controlled subpixel detector shifts. PAMI 27 (2005)
Bub, G., Tecza, M., Helmes, M., Lee, P., Kohl, P.: Temporal pixel multiplexing for simultaneous high-speed, high-resolution imaging. Nature Methods (2010)
Gu, J., Hitomi, Y., Mitsunaga, T., Nayar, S.K.: Coded rolling shutter photography: Flexible space-time sampling. In: ICCP (2010)
Agrawal, A., Veeraraghavan, A., Raskar, R.: Reinterpretable imager: Towards variable post capture space, angle & time resolution in photography. In: Eurographics (2010)
Gupta, A., Bhat, P., Dontcheva, M., Curless, B., Deussen, O., Cohen, M.: Enhancing and experiencing spacetime resolution with videos and stills. In: ICCP (2009)
Ben-Ezra, M., Nayar, S.K.: Motion-based motion deblurring. PAMI 26 (2004)
Wilburn, B., Joshi, N., Vaish, V., Levoy, M., Horowitz, M.: High speed video using a dense camera array. In: IEEE CVPR (2004)
Shechtman, E., Caspi, Y., Irani, M.: Space-time super-resolution. PAMI 27 (2005)
Agrawal, A., Gupta, M., Veeraraghavan, A., Narasimhan, S.G.: Optimal coded sampling for temporal super-resolution. In: IEEE CVPR (2010)
Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE PAMI 12 (1990)
Tschumperle, D., Deriche, R.: Vector-valued image regularization with pdes: A common framework for different applications. PAMI 27 (2005)
Project web-page, http://graphics.cs.cmu.edu/projects/FlexibleVoxels/
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)
Raskar, R., Agrawal, A., Tumblin, J.: Coded exposure photography: motion deblurring using fluttered shutter. ACM Trans. Graphics 25, 795–804 (2006)
External trigger modes supported by point grey cameras, http://www.ptgrey.com/support/kb/
Nayar, S.K., Branzoi, V., Boult, T.E.: Programmable imaging: Towards a flexible camera. In: IJCV (2006)
McDowall, I., Bolas, M.: Fast light for display, sensing and control applications. In: IEEE VR 2005 Workshop on Emerging Display Technologies (March 2005)
Schechner, Y., Nayar, S.K., Belhumeur, P.: A theory of multiplexed illumination. In: ICCV (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gupta, M., Agrawal, A., Veeraraghavan, A., Narasimhan, S.G. (2010). Flexible Voxels for Motion-Aware Videography. In: Daniilidis, K., Maragos, P., Paragios, N. (eds) Computer Vision – ECCV 2010. ECCV 2010. Lecture Notes in Computer Science, vol 6311. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15549-9_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-15549-9_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15548-2
Online ISBN: 978-3-642-15549-9
eBook Packages: Computer ScienceComputer Science (R0)