Advertisement

Low-Cost Super-Resolution Algorithms Implementation Over a HW/SW Video Compression Platform

  • Gustavo M. CallicóEmail author
  • Rafael Peset Llopis
  • Sebastian López
  • José Fco López
  • Antonio Núñez
  • Ramanathan Sethuraman
  • Roberto Sarmiento
Open Access
Research Article
First Online:
Part of the following topical collections:
  1. Super-Resolution Imaging: Analysis, Algorithms, and Applications

Abstract

Two approaches are presented in this paper to improve the quality of digital images over the sensor resolution using super-resolution techniques: iterative super-resolution (ISR) and noniterative super-resolution (NISR) algorithms. The results show important improvements in the image quality, assuming that sufficient sample data and a reasonable amount of aliasing are available at the input images. These super-resolution algorithms have been implemented over a codesign video compression platform developed by Philips Research, performing minimal changes on the overall hardware architecture. In this way, a novel and feasible low-cost implementation has been obtained by using the resources encountered in a generic hybrid video encoder. Although a specific video codec platform has been used, the methodology presented in this paper is easily extendable to any other video encoder architectures. Finally a comparison in terms of memory, computational load, and image quality for both algorithms, as well as some general statements about the final impact of the sampling process on the quality of the super-resolved (SR) image, are also presented.

Keywords

Image Quality Input Image Computational Load Aliasing Generic Hybrid 
Download to read the full article text

References

  1. 1.
    Komatsu T, Igarashi T, Aizawa K, Saito T: Very high resolution imaging scheme with multiple different-aperture cameras. Signal Processing: Image Communication 1993, 5(5–6):511–526. 10.1016/0923-5965(93)90014-KGoogle Scholar
  2. 2.
    Ghiglia DC: Space-invariant deblurring given N independently blurred images of a common object. Journal of Optical Society of America A 1984, 1(4):398–402. 10.1364/JOSAA.1.000398CrossRefGoogle Scholar
  3. 3.
    Kim SP, Su W-Y: Recursive high-resolution reconstruction of blurred multiframe images. IEEE Transactions on Image Processing 1993, 2(4):534–539. 10.1109/83.242363CrossRefGoogle Scholar
  4. 4.
    Cheeseman P, Kanefsky B, Kraft R, Stutz J, Hanson R: Super-resolved surface reconstruction from multiple images. In Maximum Entropy and Bayesian Methods. Edited by: Heidbreder GR. Kluwer Academic, Dordrecht, The Netherlands; 1996:293–308.CrossRefGoogle Scholar
  5. 5.
    Avrin V, Dinstein I: Restoration and resolution enhancement of video sequences. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97), April 1997, Munich, Germany 4: 2549–2552.Google Scholar
  6. 6.
    Eggleston P: Quality photos from digital video: salient Stills' algorithms provide a missing link. Advanced Imaging Magazine 2000, 15(5):39–41.Google Scholar
  7. 7.
    Schultz RR: Super-resolution enhancement of native digital video versus digitized NTSC sequences. Proceedings of the 5th IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI '02), April 2002, Sante Fe, NM, USA 193–197.CrossRefGoogle Scholar
  8. 8.
    Sale D, Schultz RR, Szczerba RJ: Super-resolution enhancement of night vision image sequences. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, October 2000, Nashville, Tenn, USA 3: 1633–1638.CrossRefGoogle Scholar
  9. 9.
    Gunturk BK, Batur AU, Altunbasak Y, Hayes MH III, Mersereau RM: Eigenface-based super-resolution for face recognition. Proceedings of the IEEE International Conference on Image Processing (ICIP '02), September 2002, Rochester, NY, USA 2: 845–848.Google Scholar
  10. 10.
    Capel D, Zisserman A: Super-resolution enhancement of text image sequences. Proceedings of the 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1: 600–605.CrossRefGoogle Scholar
  11. 11.
    Goyette JA, Lapin GD, Kang MG, Katsaggelos AK: Improving autoradiograph resolution using image restoration techniques. IEEE Engineering in Medicine and Biology Magazine 1994, 13(3):571–574.Google Scholar
  12. 12.
    Katsaggelos AK, Galatsanos NP (Eds): Signal Recovery Techniques for Image and Video Compression and Transmission. Kluwer Academic, Dordrecht, The Netherlands; 1998.Google Scholar
  13. 13.
    Chen D, Schultz RR: Extraction of high-resolution video stills from MPEG image sequences. Proceedings of the IEEE International Conference on Image Processing (ICIP '98), October 1998, Chicago, Ill, USA 2: 465–469.CrossRefGoogle Scholar
  14. 14.
    Erickson KJ, Schultz RR: MPEG-1 super-resolution decoding for the analysis of video still images. Proceedings of the 4th IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI '00), April 2000, Austin, Tex, USA 13–17.CrossRefGoogle Scholar
  15. 15.
    Martins B, Forchhammer S: A unified approach to restoration, deinterlacing and resolution enhancement in decoding MPEG-2 video. IEEE Transactions on Circuits and Systems for Video Technology 2002, 12(9):803–811. 10.1109/TCSVT.2002.803227CrossRefGoogle Scholar
  16. 16.
    Altunbasak Y, Patti AJ, Mersereau RM: Super-resolution still and video reconstruction from MPEG-coded video. IEEE Transactions on Circuits and Systems for Video Technology 2002, 12(4):217–226. 10.1109/76.999200CrossRefGoogle Scholar
  17. 17.
    Segall CA, Molina R, Katsaggelos AK: High-resolution images from low-resolution compressed video. IEEE Signal Processing Magazine 2003, 20(3):37–48. 10.1109/MSP.2003.1203208CrossRefGoogle Scholar
  18. 18.
    Candocia FM, Principe JC: Method using multiple models to superresolve SAR imagery. Algorithms for Synthetic Aperture Radar Imagery V, April 1998, Orlando, Fla, USA, Proceedings of SPIE 3370: 197–207.CrossRefGoogle Scholar
  19. 19.
    Cheng Y, Lu Y, Lin Z: A super resolution SAR imaging method based on CSA. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS '02), June 2002, Toronto, Canada 6: 3671–3673.CrossRefGoogle Scholar
  20. 20.
    Pastina D, Lombardo P, Farina A, Daddi P: Super-resolution of polarimetric SAR images of a ship. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS '01), July 2001, Sydney, NSW, Australia 5: 2343–2345.zbMATHGoogle Scholar
  21. 21.
    Wu R, Li J: Autofocus and super-resolution synthetic aperture radar image formation. IEE Proceedings—Radar, Sonar and Navigation 2000, 147(5):217–223. 10.1049/ip-rsn:20000616CrossRefGoogle Scholar
  22. 22.
    Yamada H, Yamaguchi Y, Rodriguez E, Kim Y, Boerner WM: Polarimetric SAR interferometry for forest canopy analysis by using the super-resolution method. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS '01), July 2001, Sydney, NSW, Australia 3: 1101–1103.Google Scholar
  23. 23.
    Liang X, Tao R, Zhou S, Wang Y: Multidimensional real aperture radar imaging. Proceedings of CIE International Conference on Radar, October 2001, Beijing, China 675–678.Google Scholar
  24. 24.
    Tatem AJ, Lewis HG, Atkinson PM, Nixon MS: Super-resolution mapping of multiple-scale land cover features using a Hopfield neural network. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS '01) , July 2001, Sydney, NSW, Australia 7: 3200–3202.Google Scholar
  25. 25.
    Freeman WT, Jones TR, Pasztor EC: Example-based super-resolution. IEEE Computer Graphics and Applications 2002, 22(2):56–65. 10.1109/38.988747CrossRefGoogle Scholar
  26. 26.
    Nagahara H, Yagi Y, Yachida M: Super-resolution from an omnidirectional image sequence. Proceedings of the 26th Annual Conference of the IEEE Industrial Electronics Society (IECON '00), October 2000, Nagoya, Japan 4: 2559–2564.Google Scholar
  27. 27.
    Nagahara H, Yagi Y, Yachida M: Resolution improving method for a 3D environment modeling using omnidirectional image sensor. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '02), May 2002, Washington, DC, USA 1: 900–907.Google Scholar
  28. 28.
    Elad M, Feuer A: Super-resolution reconstruction of image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 1999, 21(9):817–834. 10.1109/34.790425CrossRefGoogle Scholar
  29. 29.
    Elad M, Feuer A: Superresolution restoration of an image sequence: adaptive filtering approach. IEEE Transactions on Image Processing 1999, 8(3):387–395. 10.1109/83.748893CrossRefGoogle Scholar
  30. 30.
    Huang TS, Tsay RY: Multiple frame image restoration and registration. In Advances in Computer Vision and Image Processing. Volume 1. Edited by: Huang TS. JAI Press, Greenwich, Conn, USA; 1984:317–339.Google Scholar
  31. 31.
    Yen JL: On nonuniform sampling of bandwidth-limited signals. IRE Transactions on Circuit Theory 1956, 3(4):251–257.CrossRefGoogle Scholar
  32. 32.
    Papoulis A: Generalized sampling expansion. IEEE Transactions on Circuits and Systems 1977, 24(11):652–654. 10.1109/TCS.1977.1084284MathSciNetCrossRefGoogle Scholar
  33. 33.
    Huang TS (Ed): Image Sequence Processing and Dynamic Scene Analysis. Springer, Berlin, Germany; 1983.Google Scholar
  34. 34.
    Hunt BR, Kubler O: Karhunen-Loeve multispectral image restoration, part I: theory. IEEE Transactions on Acoustics, Speech, Signal Processing 1984, 32(3):592–600. 10.1109/TASSP.1984.1164363CrossRefGoogle Scholar
  35. 35.
    Zervakis ME: Optimal restoration of multichannel images based on constrained mean-square estimation. Journal of Visual Communication and Image Representation 1992, 3(4):392–411. 10.1016/1047-3203(92)90042-RMathSciNetCrossRefGoogle Scholar
  36. 36.
    Katsaggelos AK: A multiple input image restoration approach. Journal of Visual Communication and Image Representation 1990, 1(1):93–103. 10.1016/1047-3203(90)90019-RCrossRefGoogle Scholar
  37. 37.
    Katsaggelos AK, Driessen JN, Efstratiadis SN, Lagendijk RL: Spatiotemporal motion-compensated noise filtering of image sequences. Visual Communications and Image Processing IV, November 1989, Philadelphia, Pa, USA, Proceedings of SPIE 1199: 61–70.CrossRefGoogle Scholar
  38. 38.
    Katsaggelos AK, Kleihorst RP, Efstratiadis SN, Lagendijk RL: Adaptive image sequence noise filtering methods. Visual Communications and Image Processing '91: Image Processing, November 1991, Boston, Mass, USA, Proceedings of SPIE 1606: 716–727.Google Scholar
  39. 39.
    Ho S-J, Lee YH: Nonlinear spatio-temporal noise suppression techniques with applications in image sequence processing. Proceedings of the IEEE International Sympoisum on Circuits and Systems (ISCAS '91), June 1991, Singapore 1: 662–665.Google Scholar
  40. 40.
    Patti AJ, Tekalp AM, Sezan MI: Image sequence restoration and deinterlacing by motion-compensated Kalman filtering. Image and Video Processing, February 1993, San Jose, Calif, USA, Proceedings of SPIE 1903: 59–70.CrossRefGoogle Scholar
  41. 41.
    Patti AJ, Sezan MI, Tekalp AM: High-resolution image reconstruction from a low-resolution image sequence in the presence of time-varying motion blur. Proceedings of the IEEE International Conference on Image Processing (ICIP '94), November 1994, Austin, Tex, USA 1: 343–347.CrossRefGoogle Scholar
  42. 42.
    Elad M, Feuer A: Recursive optical flow estimation-adaptive filtering approach. Proceedings of the 19th IEEE Convention of Electrical and Electronics Engineers in Israel, November 1996, Jerusalem, Israel 387–390.CrossRefGoogle Scholar
  43. 43.
    Schultz RR, Stevenson RL: Improved definition video frame enhancement. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '95), May 1995, Detroit, Mich, USA 4: 2169–2172.Google Scholar
  44. 44.
    Patti AJ, Sezan MI, Tekalp AM: High resolution standards conversion of low resolution video. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '95), May 1995, Detroit, Mich, USA 4: 2197–2200.Google Scholar
  45. 45.
    Schultz RR, Stevenson RL: Extraction of high-resolution frames from video sequences. IEEE Transactions on Image Processing 1996, 5(6):996–1011. 10.1109/83.503915CrossRefGoogle Scholar
  46. 46.
    Elad M, Feuer A: Restoration of single super resolution image from several blurred, noisy, and undersampled measured images. IEEE Transactions on Image Processing 1997, 6(12):1646–1658. 10.1109/83.650118CrossRefGoogle Scholar
  47. 47.
    Kang MG, Katsaggelos AK: Simultaneous iterative image restoration and evaluation of the regularization parameter. IEEE Transactions on Signal Processing 1992, 40(9):2329–2334. 10.1109/78.157234CrossRefGoogle Scholar
  48. 48.
    Lagendijk RL, Biemond J: Iterative Identification and Restoration of Images. Kluwer Academic, Boston, Mass, USA; 1991.CrossRefGoogle Scholar
  49. 49.
    Jain AK: Fundamentals of Digital Image Processing. Prentice-Hall, Upper Saddle River, NJ, USA; 1989.zbMATHGoogle Scholar
  50. 50.
    Pratt WK: Digital Image Processing. John Wiley & Sons, New York, NY, USA; 1991.zbMATHGoogle Scholar
  51. 51.
    Zomet A, Rav-Acha A, Peleg S: Robust super-resolution. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai Marriott, Hawaii, USA 1: 645–650.Google Scholar
  52. 52.
    Youla DC: Generalized image restoration by the method of alternating orthogonal projections. IEEE Transactions on Circuits and Systems 1978, 25(9):694–702. 10.1109/TCS.1978.1084541MathSciNetCrossRefGoogle Scholar
  53. 53.
    Zhao W, Sawhney H, Hansen M, Samarasekera S: Super-fusion: a super-resolution method based on fusion. Proceedings of the 16th International Conference on Pattern Recognition (ICPR '02), August 2002, Quebec, Canada 2: 269–272.CrossRefGoogle Scholar
  54. 54.
    Irani M, Peleg S: Motion analysis for image enhancement: resolution, occlusion, and transparency. Journal of Visual Communication and Image Representation 1993, 4(4):324–335. 10.1006/jvci.1993.1030CrossRefGoogle Scholar
  55. 55.
    Freeman WT, Pasztor EC: Learning low-level vision. Proceedings of the 7th IEEE International Conference on Computer Vision (ICCV '99), September 1999, Kerkyra, Greece 2: 1182–1189.Google Scholar
  56. 56.
    Capel D, Zisserman A: Super-resolution from multiple views using learnt image models. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai Marriott, Hawaii, USA 2: 627–634.Google Scholar
  57. 57.
    Baker S, Kanade T: Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(9):1167–1183. 10.1109/TPAMI.2002.1033210CrossRefGoogle Scholar
  58. 58.
    Chaudhuri S (Ed): Super-Resolution Imaging. Kluwer Academic, Boston, Mass, USA; 2001.Google Scholar
  59. 59.
    Farsiu S, Robinson D, Elad M, Milanfar P: Fast and robust multi-frame super-resolution. IEEE Transactions on Image Processing 2004, 13(10):1327–1344. 10.1109/TIP.2004.834669CrossRefGoogle Scholar
  60. 60.
    de Beeck MO, Kleihorst RP: Super-resolution of regions of interest in a hybrid video encoder. Proceedings of Philips Conference on Digital Signal Processing (DSP '99), November 1999, Veldhoven, The NetherlandsGoogle Scholar
  61. 61.
    Callicó GM, Llopis RP, Núñez A, Sethuraman R: Mapping of real-time and low-cost super-resolution algorithms onto a hybrid video encoder. VLSI Circuits and Systems, May 2003, Maspalomas, Spain, Proceedings of SPIE 5117: 42–52.CrossRefGoogle Scholar
  62. 62.
    Llopis RP, Sethuraman R, Pinto CA, Peters H, Maul S, Oosterhuis M: A low-cost and low-power multi-standard video encoder. Proceedings of the 1st IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis, October 2003, Newport Beach, Calif, USA 97–102.Google Scholar
  63. 63.
    Callicó GM, Núñez A, Llopis RP, Sethuraman R, de Beeck MO: A low-cost implementation of super-resolution based on a video encoder. Proceedings of the 28th Annual Conference of the IEEE Industrial Electronics Society (IECON '02), November 2002, Sevilla, Spain 2: 1439–1444.Google Scholar
  64. 64.
    Llopis RP, Oosterhuis M, Ramanathan S, et al.: HW-SW co-design and verification of a multi-standard video and image codec. Proceedings of the 2nd IEEE International Symposium on Quality of Electronic Design (ISQED '01), March 2001, San Jose, Calif, USA 393–398.Google Scholar
  65. 65.
    Callicó GM, Núñez A, Llopis RP, Sethuraman R: Low-cost and real-time super-resolution over a video encoder IP. Proceedings of the 4th IEEE International Symposium on Quality Electronic Design (ISQED '03), March 2003, San Jose, Calif, USA 79–84.Google Scholar

Copyright information

© Callicó et al. 2006

Authors and Affiliations

  • Gustavo M. Callicó
    • 1
    Email author
  • Rafael Peset Llopis
    • 2
  • Sebastian López
    • 1
  • José Fco López
    • 1
  • Antonio Núñez
    • 1
  • Ramanathan Sethuraman
    • 3
  • Roberto Sarmiento
    • 1
  1. 1.Institute for Applied Microelectronics (IUMA)The University of Las Palmas de Gran CanariaTafira BajaSpain
  2. 2.Philips Consumer ElectronicsThe Netherlands
  3. 3.Philips Research LaboratoriesEindhovenThe Netherlands

Personalised recommendations