This paper introduces a practical algorithm for super-resolution, the process of reconstructing a high-resolution image from low-resolution input ones. The emphasis of the work is to super-resolve frames from real-world image/video sequences which may contain significant object occlusion or scene changes. As the quality of super-resolved images highly relies on the correctness of image alignment between consecutive frames, the robust optical flow method is used to accurately estimate motion between the image pair. An efficient and reliable scheme is devised to detect and discard incorrect matchings which may degrade the output quality. The usage of elliptical weighted average (EWA) filter is also introduced to model the point spread function (PSF) of acquisition system in order to improve accuracy of the model. A number of complex video sequences are tested to demonstrate the applicability and reliability of the proposed algorithm.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Borman S, Stevenson R. Spatial resolution enhancement of low-resolution image sequences: A comprehensive review with directions for future research. Technical Report, University of Notre Dame, 1998.
Tsai R, Huang T. Multiframe image restoration and registration. InAdvances in Computer Vision and Image Processing, 1984, 1: 317–339.
Tekalp A, Ozkan M, Sezan M. High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration. InProc. ICASSP’92. San Francisco, California, USA, Mar. 23–26, 1992, 3: 169–172.
Irani M, Peleg S. Improving Resolution by Image Registration.Journal of Computer Vision, Graphics, and Image Processing, 1991, 53(3): 231–239.
Irani M, Peleg S. Motion analysis for image enhancement: Resolution, occlusion, and transparency.Journal of Visual Communication and Image Representation 1993, 4(4): 324–335.
Elad M, Feuer A. Restoration of single super-resolution image from several blurred, noisy and down-sampled measured images.IEEE Trans. Image. Processing, 1997, 6(2): 1646–1658.
Patti A, Sezan M, Tekalp A. Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time.IEEE Trans. Image Processing, 1997, 6(8): 1064–1076.
Schultz R, Stevenson R. Extraction of high-resolution frames from video sequences.IEEE Trans. Image Processing, 1996, 5(6): 996–1011.
Hardie R, Barnard K, Armstrong E. Joint MAP registration and high-resolution image estimation using a sequence of undersampled images.IEEE Trans Image Processing, 1997, 6(12): 1621–1633.
Capel D, Zisserman A. Super-resolution enhancement of text image sequence. InProc. ICPR’2000, Barcelona, Spain, Sept. 3–8, 2000, 1: 885–891.
Elad M, Feuer A. Super-resolution reconstruction of image sequences.IEEE Trans. Pattern Analysis and Machine Intelligence, 1999, 21(9): 817–834.
Shechtman E, Caspi Y, Irani M. Increasing space-time resolution in video. InProc. ECCV’2002, Copenhagen, Denmark, May 27–June 2, 2002, 1: 753–768.
Lin Z, Shum H. On the fundamental limits of reconstruction-based super-resolution algorithms. InProc. CVPR’2001, Hawaii, USA, Dec. 11–13, 2001, 1: 1171–1176.
Eren P, Sezan M, Tekalp A. Robust, object-based high-resolution image reconstruction from low-resolution video.IEEE Trans. Image Processing, 1997, 6(10): 1446–1451.
Zomet A, Rav-Acha A, Peleg S. Robust Superresolution. InProc. CVPR’2001, Hawaii, USA, Dec. 11–13, 2001, 1: 645–650.
Zhao W, Sawhney H. Is super-resolution with optical flow feasible? InProc. ECCV’2002, Copenhagen, Denmark, May 27–June 2, 2002, 2002, 1: 599–613.
Liu C, Shum H, Zhang C. A two-step approach to hallucinating faces: Global parametric model and local nonparametric model. InProc. CVPR’2001, Hawaii, USA, Dec. 11–13, 1: 192–198.
Baker S, Kanade T. Limits on super-resolution and how to break them.IEEE Trans. Pattern Analysis and Machine Intelligence, 2002, 24(9): 1167–1183.
Hertzmann A, Jacobs C, Oliver Net al. Image analogies. InProc. SIGGRAPH’2001, Los Angeles, California, USA, August 12–17, 2001, pp.327–340.
Freeman W, Jones T, Pasztor E. Example-based super-resolution.IEEE Computer Graphics and Applications, 2002, 22(2): 56–65.
Papoulis A. Generalized sampling expansion.IEEE Trans. Circuits and Systems, 1977, 24(11): 652–654.
Yen L. On-nonuniform sampling of bandwidth-limited signals.IRE Trans. Circuits Theory, 1956, 3(4): 251–257.
Horn B. Robot Vision. MIT Press, 1986.
Barron J, Fleet D, Beauchemin S. Performance of optical flow.International Journal of Computer Vision, 1994, 12(1): 43–77.
Black M, Anandan P. The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields.Journal of Computer Vision and Image Understanding, 1996, 63(1): 75–104.
Black M, Fleet D, Yacoob Y. Robustly estimating changes in image appearance.Journal of Computer Vision and Image Understanding, 2000, 78(1): 8–31.
Greene N, Heckbert P. Creating raster ominimax images from multiple perspective views using the elliptical weighted average filter.IEEE Computer Graphics and Applications, 1986, 6(6): 21–27.
Heckbert P. Fundamentals of texture mapping and image warping [Thesis]. Department of Electrical Engineering and Computer Science, University of Califorrnia at Berkeley, June 17, 1989.
Zwicker M, Pfister H, Baar J, Gross M. Surface splatting. InProc SIGGRAPH’2001, Los Angeles, California, USA, August 12–17, 2001, pp.371–378.
Ren L, Pfister H, Zwicker M. Object space EWA surface splatting: A hardware accelerated approach to high quality point rendering.Computer Graphics Forum, 2002, 21(3): 461–470.
Zwicker M, Pfister H, Baar J, Gross M. EWA volume splatting. InProc. IEEE Visualization’2001, San Diego, CA, USA Oct. 24–26, 2001, pp.29–36.
Sawhney H, Guo Y, Hanna Ket al. Hybrid sterco camera: An IBR approach for synthesis of very high resolution stereoscopic image sequences. InProc. SIGGRAPH’2001, Los Angeles, California, USA, August 12–17, 2001, pp.451–460.
Westover L. Footprint evaluation for volume rendering. InProc. SIGGRAPH’90, Dallas, Texas, USA, Sept. 1990, pp.367–376.
Brown L. A survey of image registration techniques.ACM Computing Surveys, 1992, 24(4): 325–376.
Schreiber W. Fundamentals of Electronic Imaging Systems. Springer-Verlag, 1986.
Altunbasak Y, Patti A, Mersereau R. Super-resolution still and video reconstruction from MPEG-coded video.IEEE Trans. Circuits and Systems for Video Technology, 2002, 12(4): 217–226.
This work is supported by the National Natural Science Foundation of China (Grant Nos.69925204, 60021201, 60173035, 60103017) and the National Grand Fundamental Research 973 Program of China (Grant No.2002CB312104).
About this article
Cite this article
Jiang, Z., Lin, H., Bao, H. et al. A super-resolution method with EWA. J. Comput. Sci. & Technol. 18, 822–832 (2003). https://doi.org/10.1007/BF02945472
- EWA filter
- image-based rendering