Skip to main content
SpringerLink
Log in

Error Concealment in Encoded Video Streams

  • Chapter
Signal Recovery Techniques for Image and Video Compression and Transmission

Abstract

In ATM networks cell loss causes data to be dropped, which results in the loss of entire macroblocks when MPEG video is being transmitted. In order to reconstruct the missing data, the location of these macroblocks must be known. We describe a technique for packing ATM cells with compressed data, with the aim of detecting the location of missing macroblocks in the encoded video stream. This technique also permits proper decoding of correctly received macroblocks, and thus prevents the loss of ATM cells from affecting the decoding process. We also describe spatial and temporal techniques for the recovery of lost macroblocks. The spatial techniques fall into two categories: deterministic and statistical. A deterministic spatial approach we provide aims at reconstructing each lost pixel by spatial interpolation from the nearest undamaged pixels. Another, recovers lost macroblocks by minimizing inter-sample variations within each block and across its boundaries. In the statistical approach, each frame is modeled as a Markov Random Field, and a maximum a posteriori (MAP) estimate of the missing macroblocks is obtained based on this model. The MAP estimate for each pixel within a lost macroblock is obtained by means of the iterative conditional modes (ICM) algorithm. The iterative method is guaranteed to converge to a global maximum, even though the global maximum is not unique. It is shown that, for each pixel, the median of its neighbors is a MAP estimate. In temporal reconstruction, a search is carried out over a reference frame for the macroblock sized region that will maximize the posterior distribution of the lost macroblock given its neighbors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. J. Watkinson, The art of digital video. Oxford, England: Focal Press, second ed., 1994.

    Google Scholar 

  2. C. P. Sandbank, Digital Television. John Wiley and Sons, 1992.

    Google Scholar 

  3. A. M. Tekalp, Digital Video Processing. Prentice Hall, 1995.

    Google Scholar 

  4. A. A. Rodriguez, C. E. Fogg, and E. J. Delp, “Video compression for multimedia applications,” in Image Technology: Advances in Image Processing, Multimedia and Machine Vision, pp. 613–679, Springer-Verlag, 1996.

    Google Scholar 

  5. MPEG-1 Standard, ISO/IEC 11172, Information Technology—coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbits/s. ISO, 1993.

    Google Scholar 

  6. MPEG-2 Standard, ISO/IEC 13818, Generic Coding of Moving Pictures and Associated Audio Information. ISO, 1995.

    Google Scholar 

  7. D. LeGall, “MPEG: A video compression standard for multimedia applications,” Communications of the ACM, vol. 34, no. 4, pp. 47–58, April 1991.

    MathSciNet  Google Scholar 

  8. V. Bhaskaran and K. Konstantinides, Image and Video Compression Standards: Algorithms and Architectures. Kluwer Academic Publisher, 1995.

    Google Scholar 

  9. J. L. Black, W. B. Pennebaker, C. E. Fogg, and D. J. LeGall, MPEG Video Compression Standard. Digital Multimedia Standards Series, Chapman and Hall, 1996.

    Google Scholar 

  10. K. R. Rao and J. J. Hwang, Techniques and Standards for Image, Video and Audio Coding. Prentice Hall, 1996.

    Google Scholar 

  11. Requirements, Audio, DMIF, SNHC, Systems, Video, “Overview of the MPEG4 standard.” Stockholm meeting, document ISO/IEC JTC1/SC29/WG11 N1730, July 1997, July 1997.

    Google Scholar 

  12. MPEG Requirements Group, “MPEG-7: Context and objectives (version 4).” Stockholm meeting, document ISO/IEC JTC1/SC29/WG11 N1733, July 1997, July 1997.

    Google Scholar 

  13. A. K. Jain, Fundamentals of digital image processing. Prentice Hall Information and System Sciences Series, Prentice Hall, 1989.

    MATH  Google Scholar 

  14. MPEG-1 Video Coding Standard, ISO/IEC 11172–2, Information Technology—coding of moving pictures and associated audio for digital storage media at up to about 1. 5 Mbits/s—Part2: Video. ISO,1993.

    Google Scholar 

  15. J. R. Jain and A. K. Jain, “Displacement measurement and its application in interframe image coding,” IEEE Transactions on Communications, vol. COM29, no. 12, pp. 1799–1808, December 1981.

    Article  Google Scholar 

  16. MPEG-1 Systems Standard, ISO/IEC 11172–1, Information Technology—coding of moving pictures and associated audio for digital storage media at up to about 1. 5 Mbits/s—Partl: Systems. ISO,1993.

    Google Scholar 

  17. MPEG-2 Systems Standard, ISO/IEC 13818–1, Generic Coding of Moving Pictures and Associated Audio Information—Part1: Systems. ISO,1995.

    Google Scholar 

  18. B. G. Haskell, A. Puri, and A. N. Netravali, Digital Video: An Introduction to MPEG-2. Digital Multimedia Standards Series, Chapman and Hall, 1997.

    Google Scholar 

  19. G. C. Kessler and P. Southwick, ISDN Concepts, Facilities and Services. McGraw-Hill Series on Computer Communications, McGraw-Hill, third ed., 1997.

    Google Scholar 

  20. U. Black, ATM: Foundation for broadband networks. Prentice Hall Series in Advanced Communications Technology, Prentice Hall, 1995.

    Google Scholar 

  21. V. Kumar, Broadband communications. McGraw-Hill Series on Computer Communications, McGraw-Hill, 1995.

    Google Scholar 

  22. L. G. Cuthbert and J.-C. Sapanel, ATM: The broadband telecommunications solution. IEE Telecommunications Series 29, IEE, 1993.

    Google Scholar 

  23. M. Wada, “Selective recovery of video packet loss using error concealment,” IEEE Journal on Selected Areas in Communication, vol. 7, no. 5, pp. 807–814, June 1989.

    Article  Google Scholar 

  24. F. Kishino, K. Matanabe, Y. Hayashi, and H. Yasuda, “Variable bit rate coding of video signals for ATM networks,” IEEE Journal on Selected Areas in Communications, vol. 7, no. 5, pp. 801–806, June 1989.

    Article  Google Scholar 

  25. M. Ghanbari and V. Seferidis, “Cell-loss concealment in ATM networks,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 3, no. 3, pp. 238–247, June 1993.

    Google Scholar 

  26. M. Ghanbari and C. Hughes, “Packing coded video signals into ATM cells,” IEEE/ACM Transactions on Networking, vol. 1, no. 5, pp. 505–508, October 1993.

    Article  Google Scholar 

  27. P. Pancha and M. El Zarki, “MPEG coding for variable bit rate video transmission,” IEEE Communications Magazine, vol. 32, no. 5, pp. 54–66, May 1994.

    Article  Google Scholar 

  28. W. Luo and M. El Zarki, “Analysis of error concealment schemes for MPEG-2 video transmission over ATM based networks,” Proceedings of the SPIE Conference on Visual Communications and Image Processing,vol. 2501/3, May 1995, Taipei, Taiwan, pp. 1358–1368.

    Google Scholar 

  29. L. H. Kieu and K. N. Ngan, “Cell loss concealment techniques for layered video codec in an ATM network,” Computers and Graphics: Image Communication, vol. 18, no. 1, pp. 11–19, January-February 1994.

    Article  MathSciNet  Google Scholar 

  30. D. Raychaudhuri, H. Sun, and R. S. Girons, “ATM transport and cell-loss concealment techniques for MPEG video,” Proceedings of the International Conference on Acoustics, Speech and Signal Processing, November 1993, Minneapolis, Minnesota, pp. 117–120.

    Google Scholar 

  31. A. S. Tom, C. L. Yeh, and F. Chu, “Packet video for cell loss protection using deinterleaving and scrambling,” Proceedings of the International Conference on Acoustics, Speech and Signal Processing, May 1991, Toronto, Canada, pp. 2857–2860.

    Google Scholar 

  32. Q. Zhu, Y. Wang, and L. Shaw, “Coding and cell loss recovery in DCT based packet video,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 3, no. 3, pp. 248–258, June 1993.

    Article  Google Scholar 

  33. J.Y. Park, M.H. Lee, and K.J. Lee, “A simple concealment for ATM bursty cell loss,” IEEE Transactions on Consumer Electronics, vol. 39, no. 3, pp. 704–710, August 1993.

    Article  Google Scholar 

  34. Y. Wang, Q. Zhu, and L. Shaw, “Maximally smooth image recovery in transform coding,” IEEE Transactions on Communications, vol. 41, no. 10, pp. 1544–1551, October 1993.

    Article  MATH  Google Scholar 

  35. Y. Wang and Q. Zhu, “Signal loss recovery in DCT-based image and video codecs,” Proceedings of the SPIE Conference on Visual Communications and Image Processing, vol. 1605, November 1991, Boston, Massachusetts, pp. 667–678.

    Google Scholar 

  36. L. T. Chia, D. J. Parish, and J. W. R. Griffiths, “On the treatment of video cell loss in in the transmission of motion-JPEG and JPEG images,” Computers and Graphics: Image Communication, vol. 18, no. 1, pp. 11–19, January-February 1994.

    Article  Google Scholar 

  37. H. Sun and J. Zdepski, “Adaptive error concealment algorithm for MPEG compressed video,” Proceedings of the SPIE Conference on Visual Communications and Image Processing, vol. 1818, November 1992, Boston, Massachusetts, pp. 814–824.

    Google Scholar 

  38. W. Kwok and H. Sun, “Multidirectional interpolation for spatial error concealment,” IEEE Transactions on Consumer Electronics, vol. 3, no. 39, pp. 455–460, August 1993.

    Article  Google Scholar 

  39. H. Sun and W. Kwok, “Concealment of damaged block transform coded images using projections onto convex sets,” IEEE Transactions on Image Processing, vol. 4, no. 4, pp. 470–477, April 1995.

    Article  Google Scholar 

  40. P. Salama, N. Shroff, E. J. Coyle, and E. J. Delp, “Error concealment techniques for encoded video streams,” Proceedings of the International Conference on Image Processing,October 23–26 1995, Washington, DC, pp. 9–12.

    Google Scholar 

  41. P. Salama, N. Shroff, and E. J. Delp, “A bayesian approach to error concealment in encoded video streams,” Proceedings of the International Conference on Image Processing,September 16–19 1996, Lausanne, Switzerland, pp. 49–52.

    Google Scholar 

  42. S. Aign and K. Fazel, “Error detection and concealment measures in MPEG-2 video decoder,” Proceedings of the International Workshop on HDTV, October 1994, Torino, Italy.

    Google Scholar 

  43. S. Aign and K. Fazel, “Temporal and spatial error concealment techniques for hierarchical MPEG-2 video codec,” Proceedings of the International Workshop on Communications,June 18–20 1995, Seattle, Washington, pp. 1778–1783.

    Google Scholar 

  44. S. Aign, “Error concealment enhancement by using the reliability outputs of a SOVA in MPEG-2 video decoder,” Proceedings of the URSI International Symposium on Signal, Systems, and Electronics,October 25–27 1995, San Francisco, California, pp. 59–62.

    Google Scholar 

  45. J. F. Shen and H. M. Hang, “Compressed image concealmnet and postprocessing for digital video recording,” Proceedings of the IEEE Asia-Pacific Conference on Circuits and Systems,December 5–8 1994, Taipei, Taiwan, pp. 636–641.

    Google Scholar 

  46. D. C. Youla and H. Webb, “Image restoration by the method of convex projections: Part 1 - theory,” IEEE Transactions on Medical Imaging, vol. MI-1, no. 2, pp. 81–94, October 1982.

    Google Scholar 

  47. W. Grimson, “An implementation of a computational theory of visual surface interpolation,” Computer Vision, Graphics, Image Processing, vol. 22, no. 1, pp. 39–69, April 1983.

    Article  MathSciNet  Google Scholar 

  48. S. Geman and D. Geman, “Stochastic relaxation, gibbs distributions, and the bayesian restoration of images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-6, no. 6, pp. 721–741, November 1984.

    Article  MATH  Google Scholar 

  49. R. L. Stevenson, B. E. Schmitz, and E. J. Delp, “Discontinuity preserving regularization of inverse visual problems,” IEEE Transactions on Systems Man and Cybernetics, vol. 24, no. 3, pp. 455–469, March 1994.

    Article  Google Scholar 

  50. D. Geman and G. Reynolds, “Constrained restoration and the recovery of discontinuities,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 3, pp. 367–382, March 1992.

    Article  Google Scholar 

  51. C. Bouman and K. Sauer, “A generalized gaussian image model for edge-preserving MAP estimation,” IEEE Transactions on Image Processing, vol. 2, no. 3, pp. 296–310, July 1993.

    Article  Google Scholar 

  52. J. Marroquin, S. Mitter, and T. Poggio, “Probablistic solution of ill-posed problems in computational vision,” Journal of the American Statistical Association, vol. 82, no. 397, pp. 76–89, March 1987.

    Article  MATH  Google Scholar 

  53. R. Kinderman and J. L. Snell, Markov Random Fields and their Applications, vol. 1 of Contemporary Mathematics. American Mathematical Society, 1980.

    Google Scholar 

  54. J. Besag, “Spatial interaction and the statistical analysis of lattice systems,” Journal of the Royal Statistical Society, series B, vol. 36, pp. 192–326, 1974.

    MathSciNet  MATH  Google Scholar 

  55. R. Schultz and R. L. Stevenson, “A bayesian approach to image expansion for improved definition,” IEEE Transactions on Image Processing, vol. 3, no. 3, pp. 233–241, May 1994.

    Article  Google Scholar 

  56. P. J. Huber, Robust Statistics. John Wiley & Sons, 1981.

    Google Scholar 

  57. J. Besag, “On the statistical analysis of dirty pictures,” Journal of the Royal Statistical Society, series B, vol. 48, no. 3, pp. 259–302, 1986.

    MathSciNet  MATH  Google Scholar 

  58. D. G. Luenberger, Linear and Nonlinear Programming. Addison Wesley, second ed., 1989.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Video and Image Processing Laboratory (VIPER) School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USA

    Paul Salama, Ness B. Shroff & Edward J. Delp

Authors
  1. Paul Salama
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ness B. Shroff
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edward J. Delp
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 1998 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Salama, P., Shroff, N.B., Delp, E.J. (1998). Error Concealment in Encoded Video Streams. In: Signal Recovery Techniques for Image and Video Compression and Transmission. Springer, Boston, MA. https://doi.org/10.1007/978-1-4757-6514-4_7

Download citation

  • DOI: https://doi.org/10.1007/978-1-4757-6514-4_7

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4419-5063-5

  • Online ISBN: 978-1-4757-6514-4

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation