Abstract
We introduced, in this chapter, what the definition of a super resolution is and what the key approaching methods for major super resolution algorithms are. Numerous super resolution algorithms have based on the observation model and they have followed the warp-blur sequence. But, some cases which have large movements and warp factors such as video by taking in a vehicle are worse than normal interpolation methods. We introduce the smart and robust registration algorithm with rotation and shift estimation. To reduce the registration error, this algorithm decides the optimal reference image even other super resolution algorithms discard this registration error. This algorithm follows the warp-blur observation model because the blurring parameter is much bigger than warp parameter for camera rotation and/or vibration.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
International Organization for Standardization, ISO 12233:2000 - Photography- Electronic still picture cameras - Resolution measurements (2000)
T. Komatsu, K. Aizawa, T. Igarashi, T. Saito, Signal-processing based method for acquiring very high resolution image with multiple cameras and its theoretical analysis, in Proceedings of the Institute of Electrical Engineering, vol. 140, no. 1, pt. I (1993), pp. 19–25
S. Borman, R.L. Stevenson, Spatial resolution enhancement of low-resolution image sequences-a comprehensive review with directions for future research. Technical Report, Laboratory for Image and Signal Analysis (LISA). University of Notre Dame, Notre Dame, Ind (1998). Available at http://www.nd.edu/∼sborman/publications/
S. Borman, R.L. Stevenson, Super-resolution from image sequences-a review, in Proceedings of 1998 Midwest Symposium Circuits and Systems (1999), pp. 374–378
S. Chaudhuri (ed.), Super-Resolution Imaging (Kluwer, Norwell, 2001)
H. Ur, D. Gross, Improved resolution from sub-pixel shifted pictures. CVGIP: Graph. Models Image Process. 54, 181–186 (1992)
T. Komatsu, T. Igarashi, K. Aizawa, T. Saito, Very high resolution imaging scheme with multiple different-aperture cameras. Signal Process. Image Commun. 5, 511–526 (1993)
M.S. Alam, J.G. Bognar, R.C. Hardie, B.J. Yasuda, Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames. IEEE Trans. Instrum. Meas. 49, 915–923 (2000)
N.R. Shah, A. Zakhor, Resolution enhancement of color video sequences. IEEE Trans. Image Process. 8, 879–885 (1999)
N. Nguyen, P. Milanfar, An efficient wavelet-based algorithm for image superresolution. Proc. Int. Conf. Image Process. 2, 351–354 (2000)
R.Y. Tsai, T.S. Huang, Multipleframe image restoration and registration, in Advances in Computer Vision and Image Processing (JAI Press Inc., Greenwich, 1984), pp. 317–339
S.P. Kim, N.K. Bose, H.M. Valenzuela, Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Trans. Acoust. Speech Sig. Process. 38, 1013–1027 (1990)
S.P. Kim, W.Y. Su, Recursive high-resolution reconstruction of blurred multiframe images. IEEE Trans. Image Process. 2, 534–539 (1993)
N.K. Bose, H.C. Kim, H.M. Valenzuela, Recursive implementation of total least squares algorithm for image reconstruction from noisy, undersampled multiframes, in Proceedings of the IEEE Conference on Acoustics, Speech and Signal Processing, vol. 5 (Minneapolis, 1993), pp. 269–272
S.H. Rhee, M.G. Kang, Discrete cosine transform based regularized high-resolution image reconstruction algorithm. Opt. Eng. 38(8), 1348–1356 (1999)
M.C. Hong, M.G. Kang, A.K. Katsaggelos, A regularized multichannel restoration approach for globally optimal high resolution video sequence. in SPIE VCIP, vol. 3024 (San Jose, 1997), pp. 1306–1317
M.C. Hong, M.G. Kang, A.K. Katsaggelos, An iterative weighted regularized algorithm for improving the resolution of video sequences, in Proceedings of the International Conference on Image Processing, vol. 2 (1997), pp. 474-477
M.G. Kang, Generalized multichannel image deconvolution approach and its applications. Opt. Eng. 37(11), 2953–2964 (1998)
R.C. Hardie, K.J. Barnard, J.G. Bognar, E.E. Armstrong, E.A. Watson, High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Opt. Eng. 37(1), 247–260 (1998)
N.K. Bose, S. Lertrattanapanich, J. Koo, Advances in superresolution using L-curve, in Proceedings of the International Symposium on Circuits and Systems, vol. 2 (2001), pp. 433–436
B.C. Tom, A.K. Katsaggelos, Reconstruction of a high-resolution image by simultaneous registration, restoration, and interpolation of low-resolution images, in Proceedings of 1995 IEEE International Conference on Image Processing, vol. 2, Washington (1995), pp. 539–542
R.R. Schulz, R.L. Stevenson, Extraction of high-resolution frames from video sequences. IEEE Trans. Image Process. 5, 996–1011 (1996)
R.C. Hardie, K.J. Barnard, E.E. Armstrong, Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Trans. Image Process. 6, 1621–1633 (1997)
P. Cheeseman, B. Kanefsky, R. Kraft, J. Stutz, R. Hanson, Super-resolved surface reconstruction from multiple images. NASA Ames Research Center, Moffett Field, Technical Report FIA-94-12 (1994)
H. Stark, P. Oskoui, High resolution image recovery from image-plane arrays, using convex projections. J. Opt. Soc. Am. A 6, 1715–1726 (1989)
A.M. Tekalp, M.K. Ozkan, M.I. Sezan, High-resolution image reconstruction from lower-resolution image sequences and space varying image restoration, in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), vol. 3 (San Francisco, 1992), pp. 169–172
A.J. Patti, M.I. Sezan, A.M. Tekalp, Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Trans. Image Process. 6(8), 1064–1076 (1997)
P.E. Eren, M.I. Sezan, A.M. Tekalp, Robust, object-based high-resolution image reconstruction from low-resolution video. IEEE Trans. Image Process. 6(10), 1446–1451 (1997)
A.J. Patti, Y. Altunbasak, Artifact reduction for set theoretic super resolution image reconstruction with edge adaptive constraints and higher-order interpolants. IEEE Trans. Image Process. 10(1), 179–186 (2001)
B.C. Tom, A.K. Katsaggelos, An iterative algorithm for improving the resolution of video sequences, in Proceedings of the 1996 SPIE Conference on Visual Communications and Image Processing (Orlando, 1996), pp. 1430–1438
M. Elad, A. Feuer, Restoration of a single super-resolution image from several blurred, noisy, and undersampled measured images. IEEE Trans. Image Process. 6(12), 1646–1658 (1997)
M. Irani, S. Peleg, Improving resolution by image registration. CVGIP: Graph. Models Image Process. 53, 231–239 (1991)
S. Mann, R.W. Picard, Virtual bellows: constructing high quality stills from video, in Proceedings of the IEEE International Conference on Image Processing (Austin, 1994), pp. 13–16
M. Irani, S. Peleg, Motion analysis for image enhancement resolution, occlusion, and transparency. J. Visual Commun. Image Represent. 4, 324–335 (1993)
M. Elad, A. Feuer, Superresolution restoration of an image sequence: adaptive filtering approach. IEEE Trans. Image Process. 8, 387–395 (1999)
M. Elad, A. Feuer, Super-resolution reconstruction of image sequences. IEEE Trans. Pattern Anal. Mach. Intelli. 21(9), 817–834 (1999)
M.C. Chiang, T.E. Boult, Efficient super-resolution via image warping. Image Vis. Comput. 18, 761–771 (2000)
D. Rajan, S. Chaudhuri, Generation of super-resolution images form blurred observations using an MRF model. J. Math. Imaging Vision 16, 5–15 (2002)
D. Rajan, S. Chaudhuri, Simultaneous estimation of super-resolved intensity and depth maps from low resolution defocused observations of ascene, in Proceedings of the IEEE International Conference on Computer Vision (Vancouver, 2001), pp. 113–118
D. Rajan, S. Chaudhuri, Generalized interpolation and its applications in super-resolution imaging. Image Vis. Comput. 19, 957–969 (2001)
M.V. Joshi, S. Chaudhuri, Super-resolution imaging: use of zoom as a cue, in Proceedings of the Indian Conference on Vision, Graphics and Image Processing (Ahmedabad, 2002), pp. 439–444
N.K. Bose, H.C. Kim, B. Zhou, Performance analysis of the TLS algorithm for image reconstruction from a sequence of undersampled noisy and blurred frames, in Proceedings of the ICIP-94, IEEE International Conference on Image Processing, vol. 3 (1994), pp. 571–575
M. Ng, J. Koo, N. Bose, Constrained total least squares computations for high resolution image reconstruction with multisensors. Int. J. Imaging Syst. Technol. 12, 35–42 (2002)
M.K. Ng, N.K. Bose, Analysis of displacement errors in high-resolution image reconstruction with multisensors. IEEE Trans. Circuits Syst. I 49, 806–813 (2002)
M. Park, E. Lee, J. Park, M.G. Kang, J. Kim, DCT-based high-resolution image reconstruction considering the inaccurate sub-pixel motion information. SPIE Opt. Eng. 41(2), 370–380 (2002)
W. Lim, M. Park, M.G. Kang, Spatially adaptive regularized iterative high resolution image reconstruction algorithm, in Proceedings of the VCIP2001, Photonicswest (San Jose, 2001), pp. 20–26
E.S. Lee, M.G. Kang, Regularized adaptive high-resolution image reconstruction considering inaccurate subpixel registration. IEEE Trans. Image Process. 12, 826–837 (2003)
Wirawan, P. Duhamel, H. Maitre, Multi-channel high resolution blind image restoration, in Proceedings of the IEEE ICASSP (AZ, 1989), pp. 3229–3232
N. Nguyen, P. Milanfar, G. Golub, Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Trans. Image Process. 10, 1299–1308 (2001)
N. Nguyen, P. Milanfar, G. Golub, A computationally efficient superresolution image reconstruction algorithm. IEEE Trans. Image Process. 10, 573–583 (2001)
M. Elad, Y. Hel-Or, A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur. IEEE Trans. Image Process. 10(8), 1187–1193 (2001)
M. Ng, R. Chan, T. Chan, A. Yip, Cosine transform preconditioners for high resolution image reconstruction. Linear Algebra Appl. 316, 89–104 (2000)
D.S. Messing, M.I. Sezan, Improved multi-image resolution enhancement for colour images captured by single-CCD cameras, in Proceedings of the International Conference on Image Processing, vol. 3 (2000), pp. 484–487
M.K. Ng, An efficient parallel algorithm for high resolution color image reconstruction, in Proceedings of the 7th International Conference on Parallel and Distributed Systems: Workshops (2000), pp. 547–552
B.C. Tom, A.K. Katsaggelos, Resolution enhancement of monochrome and color video using motion compensation. IEEE Trans. Image Process. 10(2), 278–287 (2001)
M. Ng, W. Kwan, High-resolution color image reconstruction with Neumann boundary conditions. Ann. Oper. Res. 103, 99–113 (2001)
D. Chen, R.R. Schultz, Extraction of high-resolution video stills from MPEG image sequences, in Proceedings of the 1998 IEEE International Conference on Image Processing, vol. 2 (1998), pp. 465–469
Y. Altunbasak, A.J. Patti, A maximum a posteriori estimator for high resolution video reconstruction from MPEG video, in Proceedings of the 2000 IEEE International Conference on Image Processing, vol. 2 (2000), pp. 649–652
B. Martins, S. Forchhammer, A unified approach to restoration, deinterlacing and resolution enhancement in decoding MPEG-2 video. IEEE Trans. Circuits Syst. Video Technol. 12(9), 803–811 (2002)
C.A. Segall, R. Molina, A.K. Katsaggelos, J. Mateos, Reconstruction of high-resolution image frames from a sequence of low-resolution and compressed observations, in Proceedings of the 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 2 (2002), pp. 1701–1704
S.C. Park, M.G. Kang, C.A. Segall, A.K. Katsaggelos, Spatially adaptive high-resolution image reconstruction of low resolution DCT-based compressed images, in Proceedings of the 2002 IEEE International Conference Image Processing, vol. 2 (2002), pp. 861–864
B.K. Gunturk, Y. Altunbasak, R.M. Mersereau, Multiframe resolution-enhancement methods for compressed video. IEEE Signal Process. Lett. 9, 170–174 (2002)
H.C. Andrews, B.R. Hunt, Digital Image Restoration (Prentice-Hall, Englewood Cliffs, 1977)
A.K. Katsaggelos (ed.), Digital Image Restoration, vol. 23 (Springer, Heidelberg, 1991)
I.J. Schoenberg, Cardinal interpolation and spline functions. J. Approx. Theory. 2, 167–206 (1969)
R.E. Crochiere, L.R. Rabiner, Interpolation and decimation of digital signals-a turorial review. Proc. IEEE 69(3), 300–331 (1981)
M. Unser, A. Aldroubi, M. Eden, Enlargement or reduction of digital images with minimum loss of information. IEEE Trans. Image Process. 4(3), 247–258 (1995)
Z. Wang, F. Qi, On ambiguities in super-resolution modeling. IEEE Signal Process. Lett. 11, 678–681 (2004)
S. Farsiu, D. Robinson, M. Elad, P. Milanfar, Robust shift and add approach to super-resolution, in Proceedings of SPIE Applications of Digital Image Processing XXVI, vol. 5203 (San Diego, 2003), pp. 121–130
S. Farsiu, D. Robinson, M. Elad, P. Milanfar, Fast and robust multiframe super-resolution. IEEE Trans. Image Process. 13, 1327–1344 (2004)
A. L´opez, R.Molina, A. K. Katsaggelos, A. Rodr´ıguez, J. M. L´opez, J. M. Llamas, Parameter estimation in Bayesian reconstruction of SPECT images: an aide in nuclear medicine diagnosis. Int. J. Imaging Syst. Technol. 14, 21–27 (2004)
S. Lertrattanapanich, N.K. Bose, High resolution image formation from low resolution frames using Delaunay triangulation. IEEE Trans. Image Process. 11, 1427–1441 (2002)
S.C. Park, M.K. Park, M.G. Kang, Super-resolution image reconstruction: a technical overview. IEEE Sig. Process. Mag. 20(3), 21–36 (2003)
B. Zitová, J. Flusser, Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003)
A.M. Tekalp, Digital Video Processing (Prentice Hall, Englewood Cliffs, 1995)
B.S. Reddy, B.N. Chatterji, An FFT-based technique for translation, rotation, and scale-invariant image registration. IEEE Trans. Image Process. 5(8), 1266–1271 (1996)
B. Marcel, M. Briot, R. Murrieta, Calcul de translation et rotation par la transformation de Fourier. Traitement du Signal 14(2), 135–149 (1997)
S.P. Kim, W.-Y. Su, Subpixel accuracy image registration by spectrum cancellation, in Proceedings of IEEE International Conference Acoustics, Speech, Signal Processing (ICASSP ’93), vol. 5 (Minneapolis, 1993), pp. 153–156
H.S. Stone, M.T. Orchard, E.-C. Chang, S.A. Martucci, A fast direct Fourier-based algorithm for subpixel registration of images. IEEE Trans. Geosci. Remote Sens. 39(10), 2235–2243 (2001)
P. Vandewalle, S.E. S¨usstrunk, M. Vetterli, Super-resolution images reconstructed from aliased images, in Proceedings of SPIE/IS&T Visual Communications and Image Processing Conference, ed. by T. Ebrahimi, T. Sikora, vol. 5150 (Lugano, 2003), pp. 1398–1405
H. Foroosh, J.B. Zerubia, M. Berthod, Extension of phase correlation to subpixel registration. IEEE Trans. Image Process. 11(3), 188–200 (2002)
L. Lucchese, G.M. Cortelazzo, A noise-robust frequency domain technique for estimating planar roto-translations. IEEE Trans. Sig. Process. 48(6), 1769–1786 (2000)
D. Capel, A. Zisserman, Computer vision applied to super-resolution. IEEE Sig. Process. Mag. 20(3), 75–86 (2003)
M.A. Fischler, R.C. Bolles, Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Comm. of the ACM 24(6), 381–395
D. Keren, S. Peleg, R. Brada, Image sequence enhancement using sub-pixel displacements, in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 88), (Ann Arbor, 1988), pp. 742–746
J.R. Bergen, P. Anandan, K.J. Hanna, R. Hingorani, Hierarchical model-based motion estimation, in Proceedings of 2nd European Conference on Computer Vision (ECCV’92), Lecture Notes in Computer Science (Santa Margherita Ligure, 1992), pp. 237–252
M. Irani, B. Rousso, S. Peleg, Computing occluding and transparent motions. Int. J. Comput. Vis. 12(1), 5–16 (1994)
J. Gluckman, Gradient field distributions for the registration of images, in Proceedings of IEEE International Conference on Image Processing (ICIP ’03), vol. 3 (Barcelona, 2003), pp. 691–694
A. Papoulis, Generalized sampling expansion. IEEE Trans. Circuits Syst. 24(11), 652–654 (1977)
R.R. Schultz, L. Meng, R.L. Stevenson, Subpixel motion estimation for super-resolution image sequence enhancement. J. Vis. Commun. Image Represent. 9(1), 38–50 (1998)
A. Zomet, A. Rav-Acha, S. Peleg, Robust super-resolution, in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR ’01), vol. 1, (Kauai, 2001), pp. 645–650
W. Li, E. Salari, Successive elimination algorithm for motion estimation. IEEE Trans. Image Process. 4(1), 105–107 (1995)
X.Q. Gao, C.J. Duanmu, C.R. Zou, A multilevel successive elimination algorithm for block matching motion estimation. IEEE Trans. Image Process. 9(3), 501–505 (2000)
M. Brűnig, W. Niehsen, Fast full search Block matching. IEEE Trans. Circuit Syst. Video Technol. 11(2), 241–247 (2001)
H.G. Musmann, P. Pirsch, H.J. Grallert, Advanced in picture coding. Proc. IEEE 73(4), 523–548 (1995)
S. Zhu, K. Ma, A new diamond search algorithm for fast block matching motion estimation. IEEE Trans. Image Process. 9, 287–290 (2000)
ITU-T, Video coding for low bitrate communication, Draft Recommendation H.263 (1995)
Y.H. Jeong, J.H. Kim, The FASCO block matching algorithm based on motion vector prediction using spatio-temporal correlations. Korean Inst. Commun. Sci. 26(11A), 1925–1937 (2002)
J.N. Kim, S.C. Byun, Y.H. Kim, B.H. Ahn, Fast full search motion estimation algorithm using early detection of impossible candidate vectors. IEEE Trans. Sig. Process. 50(9), 2355–2365 (2002)
V. Ayala Ramirez, M. Devy, C. Parra, Active tracking based on Hausdorff matching, in Proceedings of Pattern Recognition 15th International Conference, vol. 4 (2000), pp. 706–709
V. Velisavljevic, “Directionlets,” Ph.D. dissertation, Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Lausanne, Switzerland, 2005, ph.D. Thesis EPFL 3358 (2005), School of Computer and Communication Sciences
W.S. Hoge, A subspace identification extension to the phase correlation method. IEEE Trans. Med. Imaging 22(2), 277–280 (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Cho, HM. (2014). Super Resolution. In: Kim, J., Shin, H. (eds) Algorithm & SoC Design for Automotive Vision Systems. Springer, Dordrecht. https://doi.org/10.1007/978-94-017-9075-8_3
Download citation
DOI: https://doi.org/10.1007/978-94-017-9075-8_3
Published:
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-017-9074-1
Online ISBN: 978-94-017-9075-8
eBook Packages: EngineeringEngineering (R0)