Abstract
Multi-sampled imaging is a general framework for using pixels on an image detector to simultaneously sample multiple dimensions of imaging (space, time, spectrum, brightness, polarization, etc.). The mosaic of red, green and blue spectral filters found in most solid-state color cameras is one example of multi-sampled imaging. We briefly describe how multi-sampling can be used to explore other dimensions of imaging. Once such an image is captured, smooth reconstructions along the individual dimensions can be obtained using standard interpolation algorithms. Typically, this results in a substantial reduction of resolution (and hence image quality). One can extract significantly greater resolution in each dimension by noting that the light fields associated with real scenes have enormous redundancies within them, causing different dimensions to be highly correlated. Hence, multi-sampled images can be better interpolated using local structural models that are learned offline from a diverse set of training images. The specific type of structural models we use are based on polynomial functions of measured image intensities. They are very effective as well as computationally efficient. We demonstrate the benefits of structural interpolation using three specific applications. These are (a) traditional color imaging with a mosaic of color filters, (b) high dynamic range monochrome imaging using a mosaic of exposure filters, and (c) high dynamic range color imaging using a mosaic of overlapping color and exposure filters.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download to read the full chapter text
Chapter PDF
References
Authors. Resolution Enhancement Along Multiple Imaging Dimensions Using Structural Models. Technical Report, Authors’ Institution, 2001.
B. E. Bayer. Color imaging array. U.S. Patent 3,971,065, July 1976.
M. Ben-Ezra. Segmentation with Invisible Keying Signal. In CVPR, 2000.
S. Baker and T. Kanade. Limits on Super-Resolution and How to Break Them. In Proc. of CVPR, pages II:372–379, 2000.
R. N. Bracewell. The Fourier Transform and Its Applications. McGraw Hill, 1965.
D. Brainard. Bayesian method for reconstructing color images from trichromatic samples. In Proc. of IS&T 47th Annual Meeting, pages 375–380, Rochester, New York, 1994.
P. L. P. Dillon. Color imaging array. U.S. Patent 4,047203, September 1977.
P. L. P. Dillon. Fabrication and performance of color filter arrays for solid-state imagers. In IEEE Trans. on Electron Devices, volume ED-25, pages 97–101, 1978.
P. Debevec and J. Malik. Recovering High Dynamic Range Radiance Maps from Photographs. Proc. of ACM SIGGRAPH 1997, pages 369–378, 1997.
W. Freeman and E. Pasztor. Learning Low-Level Vision. In Proc. of ICCV, pages 1182–1189, 1999.
R. Ginosar, O. Hilsenrath, and Y. Zeevi. Wide dynamic range camera. U.S. Patent 5,144,442, September 1992.
K. Knop and R. Morf. A new class of mosaic color encoding patterns for single-chip cameras. In IEEE Trans. on Electron Devices, volume ED-32, 1985.
T. Mitsunaga and S. K. Nayar. Radiometric Self Calibration. In Proc. of CVPR, pages I:374–380, 1999.
D. Manabe, T. Ohta, and Y. Shimidzu. Color filter array for IC image sensor. In Proc. of IEEE Custom Integrated Circuits Conference, pages 451–455, 1983.
S. Mann and R. Picard. Being ‘Undigital’ with Digital Cameras: Extending Dynamic Range by Combining Differently Exposed Pictures. Proc. of IST’s 48th Annual Conf., pages 442–448, May 1995.
S. K. Nayar and T. Mitsunaga. High Dynamic Range Imaging: Spatially Varying Pixel Exposures. In Proc. of CVPR, pages I:472–479, 2000.
K. A. Parulski. Color filters and processing alternatives for one-chip cameras. In IEEE Trans. on Electron Devices, volume ED-32, 1985.
Y. Y. Schechner and S. K. Nayar. Generalized mosaicing. In ICCV, 2001.
M. A. Wober and R. Soini. Method and apparatus for recovering image data through the use of a color test pattern. U.S. Patent 5,475,769, Dec. 1995.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nayar, S.K., Narasimhan, S.G. (2002). Assorted Pixels: Multi-sampled Imaging with Structural Models. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds) Computer Vision — ECCV 2002. ECCV 2002. Lecture Notes in Computer Science, vol 2353. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-47979-1_43
Download citation
DOI: https://doi.org/10.1007/3-540-47979-1_43
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43748-2
Online ISBN: 978-3-540-47979-6
eBook Packages: Springer Book Archive