Multimedia Tools and Applications

, Volume 74, Issue 6, pp 2033–2054 | Cite as

Switchable video error concealment using encoder driven scene transition detection and edge preserving SEC



Error concealment, a decoder based technique, attempts to reconstruct the corrupted regions of a video frame, using spatial and/or temporal correlation. For sequences with high temporal correlations, the performance of Temporal Error Concealment (TEC) techniques are better than Spatial Error Concealment (SEC) techniques in terms of PSNR of the reconstructed frames. However, the performance of TECs deteriorate drastically at the scene boundary due to low temporal correlation with the reference frame. In this paper, we propose a novel encoder controlled transition detection scheme which would facilitate selection of concealment strategy, that makes a choice between our newly proposed edge-direction based SEC technique and a TEC scheme. Exploiting Motion Vector (MV), Sum of Absolute Difference (SAD), and Motion Compensated Histogram Difference (MCHD), scene transition is detected at the encoder. A single bit transition status flag per frame is then sent to the decoder as a part of the header information to select the concealment policy between TEC and SEC. The present work also proposes a new edge-directed spatial error concealment, termed as “Directional Edge Based Spatial Error Concealment” (DEBSEC), which outperforms existing techniques in terms of PSNR and SSIM of the concealed frame.


Scene transition Scene and illumination change index Error concealment Spatial error concealment DEBSEC H.264/AVC 


  1. 1.
    Asheri H, Rabiee HR, Pourdamghani N, Ghanbari M (2012) Multi-directional spatial error concealment using adaptive edge thresholding. IEEE Trans Consum Electron 58(3):880–885CrossRefGoogle Scholar
  2. 2.
    Berlin FH JVT H.264/MPEG-4 AVC reference software. Online available: Accessed 10 Oct 2010
  3. 3.
    Bouthemy P, Gelgon M, Ganansia F (1999) A unified approach to shot change detection and camera motion characterization. IEEE Trans Circ Syst Video Technol 9:1030–1044CrossRefGoogle Scholar
  4. 4.
    Carnec M, Le Callet P, Barba D (2003) An image quality assessment method based on perception of structural information. In: Proc. int. conf. image processing (ICIP 2003), vol 3Google Scholar
  5. 5.
    Chang PS, Chou YZ (1999) Efficient mpeg compressed video analysis using macroblock type information. IEEE Trans Multimedia 1:321–333CrossRefGoogle Scholar
  6. 6.
    Chen Y, Hu Y, Au OC, Li H, Chen CW (2008) Video error concealment using spatio-temporal boundary matching and partial differential equation. IEEE Trans Multimedia 10(1):2–15CrossRefGoogle Scholar
  7. 7.
    Chung MG, Kim H, Song SMH (2000) A scene boundary detection method. In: Proc. of the int. conf. on image processing (ICIP), vol 3, pp 933–936Google Scholar
  8. 8.
    Dimou A, Nemethova O, Rupp M (2005) Scene change detection for H.264 using dynamic threshold techniques. In: Proc. of the 5th EURASIP conference on speech, image processing, multimedia communications, and service. Slovac RepublicGoogle Scholar
  9. 9.
    Gharavi H, Gao S (2008) Spatial interpolation algorithm for error concealment. In: Proc. of the IEEE ICASSP conference, Las VegasGoogle Scholar
  10. 10.
    Guimar SJF, Couprie M (2003) Video segmentation based on 2d image analysis. Pattern Recognit Lett 24:947–953CrossRefGoogle Scholar
  11. 11.
    Hanjalic A (2002) Shot-boundary detection: unraveled and resolved? IEEE Trans Circ Syst Video Technol 12(2):90–105CrossRefGoogle Scholar
  12. 12.
    ITU Recommendation H.264 Accessed 1 Aug 2009
  13. 13.
    Jie F, Aiai H, Yaowu C (2008) A novel scene change detection algorithm for H.264/AVC bitstreams. In: Pacific-Asia workshop on computational intelligence and industrial application (PACIIA–08), vol 1, pp 712–716Google Scholar
  14. 14.
    Kang SJ, Cho SI, Yoo S, Kim YH (2013) Multi-histogram based scene change detection for frame rate up-conversion. In: IEEE international conference on consumer electronics (ICCE) 2013, pp 332–333Google Scholar
  15. 15.
    Kim W, Koo J, Jeong J (2006) Fine directional interpolation for spatial error concealment. IEEE Trans Consum Electron 52(3):1050–1056CrossRefGoogle Scholar
  16. 16.
    Kumwilaisak W, Kuo CCJ (2011) Spatial error concealment with sequence-aligned texture modeling and adaptive directional recovery. J Vis Commun Image Represent 22(2):164–177CrossRefGoogle Scholar
  17. 17.
    Kung WY, Kim CS, Kuo CCJ (2006) Spatial and temporal error concealment techniques for video transmission over noisy channels. IEEE Trans Circ Syst Video Technol 49:789–803CrossRefGoogle Scholar
  18. 18.
    Li YN, Lu ZM, Niu XM (2009) Fast video shot boundary detection framework employing pre-processing techniques. IET Image Process 3:121–134CrossRefMATHGoogle Scholar
  19. 19.
    Lo CC, Wang SJ (2001) Video segmentation using a histogram-based fuzzy c-means clustering algorithm. In: The 10th IEEE international conference on fuzzy systems 2001, vol 2, pp 920–923Google Scholar
  20. 20.
    Ma M, Au OC, Chan SHG, Sun MT (2010) Edge-directed error concealment. IEEE Trans Circ Syst Video Technol 20:382–395CrossRefGoogle Scholar
  21. 21.
    Midya A, Sengupta S (2011) Hybrid temporal/spatial error concealment strategy robust to scene transitions. In: Proc. IEEE PACRIM-11, Victoria (BC), CanadaGoogle Scholar
  22. 22.
    Nagasaka A, Tanaka Y (1991) Automatic video indexing and full-video search for object appearances. In: IFIP working conference on visual database systems, p 113–127Google Scholar
  23. 23.
    Ngo CW, Pong TC, Chin RT (2001) Video partitioning by temporal slice coherency. IEEE Trans Circ Syst Video Technol 11:941–953CrossRefGoogle Scholar
  24. 24.
    Pei SC, Chou YZ (2004) Novel error concealment method with adaptive prediction to the abrupt and gradual scene changes. IEEE Trans Multimedia 6:789–803CrossRefGoogle Scholar
  25. 25.
    Pyun JY (2008) Error concealment aware streaming video system over packet-based mobile networks. IEEE Trans Consum Electron 54(4):1708–1713CrossRefGoogle Scholar
  26. 26.
    Qaratlu MM, Ghanbari M (2011) Intra-frame loss concealment based on directional extrapolation. Signal Process Image Commun 26(6):304–309CrossRefGoogle Scholar
  27. 27.
    Tan YP, Nagamani J, Lu H (2003) Modified kolmogorov-smirnov metric for shot boundary detection. Electron Lett 39(18):1313–1315CrossRefGoogle Scholar
  28. 28.
    Taniguchi Y, Akutsu A, Tonomura Y (1997) Panorama excerpts: extracting and packing panoramas for video browsing. In: ACM international conference on multimedia, Seattle, WA, pp 427–436Google Scholar
  29. 29.
    Tonomura Y, Abe S (1990) Content oriented visual interface using video icons for visual database systems. J Vis Lang Comput 1:183–198CrossRefGoogle Scholar
  30. 30.
    Chowdhury UM, Rahman R, Sana J, Kabir SMR (2007) Fast scene change detection based histogram. In: IEEE/ACIS international conference on computer and information science, pp 229–233Google Scholar
  31. 31.
    Valente S, Dufour C, Groliere F, Snook D (2001) An efficient error concealment implementation for MPEG-4 video streams. IEEE Trans Consum Electron 47(3):568–578CrossRefGoogle Scholar
  32. 32.
    Wang Zhu QF (1998) Error control and concealment for video communication: a review. Proc IEEE 86:974–997CrossRefGoogle Scholar
  33. 33.
    Wang Y, Zhu QF, Shaw L (1993) Maximally smooth image recovery in transform coding. IEEE Trans Commun 41:1544–1551CrossRefMATHGoogle Scholar
  34. 34.
    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612CrossRefGoogle Scholar
  35. 35.
    Wiegand T, Sullivan GJ, Bjøntegaard G, Luthra A (2003) Overview of the H.264/AVC video coding standard. IEEE Trans Circ Syst Video Technol 13:560–576CrossRefGoogle Scholar
  36. 36.
    Yan B, Gharavi H (2010) A hybrid frame concealment algorithm for H.264/AVC. IEEE Trans Image Process 19:98–107CrossRefMathSciNetGoogle Scholar
  37. 37.
    Ye-Kui W, Hannuksela M, Varsa V, Hourunranta A, Gabbouj M (2002) The error concealment feature in the H.26L test model. In: Proc. of the int. conf. on image processing (ICIP), vol 2, pp II 729–II 732Google Scholar
  38. 38.
    Yeo BL, Liu B (1995) Rapid scene analysis on compressed video. IEEE Trans Circ Syst Video Technol 5:533–544CrossRefGoogle Scholar
  39. 39.
    Yoo HW, Ryoo HJ, Jang DS (2006) Gradual shot boundary detection using localized edge blocks. Multimed Tools Appl 28(3):283–300CrossRefGoogle Scholar
  40. 40.
    Zabih R, Miller J, Mai K (1995) A feature-based algorithm for detecting and classifying scene breaks. In: Proc. of ACM multimedia, pp 189–200Google Scholar
  41. 41.
    Zeng W, Liu B (1999) Geometric-structure-based error concealment with novel applications in block-based low-bit-rate coding. IEEE Trans Circ Syst Video Technol 9:648–645CrossRefGoogle Scholar
  42. 42.
    Zhao Y, Yu L, Chen Z, Zhu C (2011) Video quality assessment based on measuring perceptual noise from spatial and temporal perspectives. IEEE Trans Circ Syst Video Technol 21(12):1890–1902CrossRefGoogle Scholar
  43. 43.
    Zhu L, Qu J, Rahman MA, Hong W (2010) An integrated method for video shot boundary detection. In: Proc. of the IEEE SoutheastCon 2010, Concord, NC, pp 151–154Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Computer Vision Lab, Dept. of Electronics & Electrical Comm. Engg.Indian Institute of TechnologyKharagpurIndia
  2. 2.Dept. of Electronics & Electrical Comm. Engg.Indian Institute of TechnologyKharagpurIndia

Personalised recommendations