Multimedia Tools and Applications

, Volume 78, Issue 6, pp 7453–7477 | Cite as

Detection of motion compensated frame interpolation via motion-aligned temporal difference

  • Xiangling Ding
  • Yue Li
  • Ming Xia
  • Jiale He
  • Gaobo YangEmail author


Motion compensated frame interpolation (MCFI) is a special frame based video manipulation, which increases the temporal continuity of low frame rate videos by synthesizing new frames between successive frames. MCFI might also be used to counterfeit high frame-rate videos, which mislead users’ attraction and waste storage spaces in video-sharing websites. Existing MCFI detectors are designed to judge the absence or presence of MCFI forgery in a controllable environment of known MCFI techniques. Practical detector should consider the possibility of unknown MCFI techniques. We are motivated to propose a robust MCFI detector for more practical scenarios. Considering the effects of non-motion regions in candidate frame, the statistical moments are firstly extracted from motion-aligned frame differences (MAFD). Then, the one-class support vector machine (SVM), following a training stage capturing the properties of original frames, is exploited to judge whether the candidate frame is interpolated by MCFI or not. Finally, a special interpolated frame detection (SIFD) is designed to pick out interpolated frames, which are synthesized from two consecutive reference frames with no motion vectors (MVs) or less MVs. A series of experiments evaluated on four representative MCFI techniques have shown promising results.


Video forensics Motion compensated frame interpolation Motion-aligned One-class SVM 



The authors appreciate Dr. Ran Li in Xinyang Normal University, China for permission to use their codes in theirs experiments. They appreciate Dr. Won Hee Lee in Korea Advanced Institute of Science and Technology, Korea for providing up-converted videos in the experiments. They also appreciate Dr Ningbo Zhu in Hunan University, China for providing technical editing of the revised manuscript. This work is supported in part by the National Natural Science Foundation of China (61572183, 61379143).


  1. 1.
    Available on the online video Databases.
  2. 2.
    Available on the online video Databases.
  3. 3.
    Bestagini P, Battaglia S, Milani S, Tagliasacchi M, Tubaro S (2013) Detection of temporal interpolation in video sequences. In: Proceedings of the International Conference Acoustic, Speech Signal Process, pp 3033–3037Google Scholar
  4. 4.
    Bian S, Luo W, Huang J (2014) Detecting video frame-rate upconversion based on periodic properties of inter-frame similarity. Multimed Tools Appl 72(1):437–451CrossRefGoogle Scholar
  5. 5.
    Black MJ, Anandan P (1993) A framework for the robust estimation of optical flow. In: Proceedings of the 4th International Conference on Computing Vision, pp 231–236Google Scholar
  6. 6.
    Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. In: Proceedings of the ACM 5th Annual Workshop COLT, Pittsburgh, PA USA, pp 144–152Google Scholar
  7. 7.
    Chang X, Ma Z, Lin M et al (2017) Feature interaction augmented sparse learning for fast kinect motion detection. IEEE Trans Image Process 26(8):3911–3920MathSciNetCrossRefGoogle Scholar
  8. 8.
    Chang X, Ma Z, Yang Y et al (2017) Bi-level semantic representation analysis for multimedia event detection. IEEE Trans Cybern 47(5):1180–1197CrossRefGoogle Scholar
  9. 9.
    Chang X, Yang Y (2017) Semisupervised feature analysis by mining correlations among multiple tasks. IEEE Trans Neural Netw Learn Syst 28(10):2294–2305MathSciNetCrossRefGoogle Scholar
  10. 10.
    Chang X, Yu YL, Yang Y et al (2017) Semantic pooling for complex event analysis in untrimmed videos. IEEE Trans Pattern Anal Mach Intell 39(8):1617–1632CrossRefGoogle Scholar
  11. 11.
    Chao J, Jiang X, Sun T (2012) A novel video inter-frame forgery model detection scheme based on optical flow consistency. In: Digital forensics and watermaking. Springer, Berlin, pp 267–281CrossRefGoogle Scholar
  12. 12.
    Chen R, Yang G, Zhu N (2014) Detection of object-based manipulation by the statistical features of object contour. Forensic Sci Int 236:164–169CrossRefGoogle Scholar
  13. 13.
    Chen S, Tan S, Li B, Huang J (2016) Automatic detection of object-based forgery in advanced video. IEEE Trans Circuits Syst Video Technol 26(11):2138–2151CrossRefGoogle Scholar
  14. 14.
    Conotter V, OBrien J, Farid H (2012) Exposing digital forgeries in ballistic motion. IEEE Trans Inf Forensics Secur 7(1):283–296CrossRefGoogle Scholar
  15. 15.
    Dar Y, Bruckstein AM (2015) Motion-compensated coding and frame rate up-conversion: models and analysis. IEEE Trans Image Process 24(7):2051–2066MathSciNetzbMATHGoogle Scholar
  16. 16.
    Ding X, Yang G, Li R, Zhang L, Li Y, Sun X (2017) Identification of motion-compensated frame rate up-conversion based on residual signal. IEEE Trans Circuits Syst Video Technol. CrossRefGoogle Scholar
  17. 17.
    Kang SJ, Cho SI, Yoo S, Kim YH (2012) Scene change detection using multiple histograms for motion-compensated frame rate upconversion. J Display Technol 8(3):121–126CrossRefGoogle Scholar
  18. 18.
    Kodovsky J, Fridrich J, Holub V (2012) Ensemble classifiers for steganalysis of digital media. IEEE Trans Inf Forensics Secur 7(2):432–444CrossRefGoogle Scholar
  19. 19.
    Lee WH, Choi K, Ra JB (2014) Frame rate up conversion based on variational image fusion. IEEE Trans Image Process 23(1):399–412MathSciNetCrossRefGoogle Scholar
  20. 20.
    Li H, Luo W, Qiu X, Huang J (2016) Identification of various image operations using residual-based features. IEEE Trans Circuits Syst Video Technol. CrossRefGoogle Scholar
  21. 21.
    Li R, Gan Z, Cui Z, Tang G, Zhu X (2014) Multi-channel mixed-pattern based frame rate up-conversion using spatio-temporal motion vector refinement and dual weighted overlapped block motion compensation. J Disp Technol 10(12):1010–1023CrossRefGoogle Scholar
  22. 22.
    Li R, Liu Z, Zhang Y, Li Y, Fu Z (2018) Noise-level estimation based detection of motion-compensated frame interpolation in video sequences. Multimed Tools Appl 77(1):663–688CrossRefGoogle Scholar
  23. 23.
    Li Z, Nie F, Chang X, Yang Y (2017) Beyond trace ratio: weighted harmonic mean of trace ratios for multiclass discriminant analysis. IEEE Trans Knowl Data Eng 29(10):2100–2110CrossRefGoogle Scholar
  24. 24.
    Liu HB, Xin RQ, Zhao DB, Ma SW, Gao W (2012) Multiple hypotheses bayesian frame rate up-conversion by adaptive fusion of motion-compensated interpolations. IEEE Trans Circuits Syst Video Technol 22(8):1188–1198CrossRefGoogle Scholar
  25. 25.
    Liu Y, Huang T (2015) Exposing video inter-frame forgery by Zernike opponent chromaticity moments and coarseness analysis. Multimed Syst 23(2):223–238MathSciNetCrossRefGoogle Scholar
  26. 26.
    Milani S, Fontani M, Bestagini P, Barni M, Piva A, Tagliasacchi M, Tubaro S (2012) An overview on video forensics. APSIPA Trans Signal Inf Process 1:1–18CrossRefGoogle Scholar
  27. 27.
    Nie Y, Ma KK (2002) Adaptive rood pattern search for fast block-matching motion estimation. IEEE Trans Image Process 11(12):1442–1448CrossRefGoogle Scholar
  28. 28.
    Po LM, Ma WC (1996) A novel four-step search algorithm for fast block motion estimation. IEEE Trans Circuits Syst Video Technol 6(3):313–317CrossRefGoogle Scholar
  29. 29.
    Rijsbergen CJV (1979) Information retrieval. Butterworth-Heinemann, NewtonzbMATHGoogle Scholar
  30. 30.
    Rocha A, Scheirer W, Boult TE, Goldenstein S (2011) Vision of the unseen: current trends and challenges in digital image and video forensics. ACM Comput Surveys 43:1–42CrossRefGoogle Scholar
  31. 31.
    Stamm MC, Lin W, Liu KJ (2012) Temporal forensics and anti-forensics for motion compensated video. IEEE Trans Inf Forensics Security 7(4):1315–1329CrossRefGoogle Scholar
  32. 32.
    Stamm MC, Wu M, Liu KJ (2013) Information forensics: an overview of the first decade. IEEE Access 1:167–200CrossRefGoogle Scholar
  33. 33.
    Vapnik VN, Lerner A (1963) Pattern recognition using generalized portrait method. Autom Remote Control 24(6):774–780Google Scholar
  34. 34.
    Wang T, Snoussi H (2014) Detection of abnormal visual events via global optical flow orientation histogram. IEEE Trans Inf Forensics Secur 9(6):988–998CrossRefGoogle Scholar
  35. 35.
    Wang W, Farid H (2007) Exposing digital forgeries in interlaced and deinterlaced video. IEEE Trans Inf Forensics Security 2(3):438–449CrossRefGoogle Scholar
  36. 36.
    Wu Y, Jiang X, Sun T, Wang W (2014) Exposing video inter-frame forgery based on velocity field consistency. In: Proceedings of the International Conference Acoustic, Speech Signal Process, pp 2674–2678Google Scholar
  37. 37.
    Xia M, Yang G, Li L, Li R, Sun X (2016) Detecting video frame rate up-conversion based on frame-level analysis of average texture variation. Multimed Tools Appl:1–23Google Scholar
  38. 38.
    Yao Y, Yang G, Sun X, Li L (2016) Detecting video frame-rate up-conversion based on periodic properties of edge-intensity. Journal of Information Security and Applications 26:39–50CrossRefGoogle Scholar
  39. 39.
    Yoo DG, Kang SJ, Kim YH (2013) Direction-select motion estimation for motion-compensated frame rate up-conversion. J Display Technol 9(10):840–850CrossRefGoogle Scholar
  40. 40.
    Zhang J, Su Y, Zhang M (2009) Exposing digital video forgery by ghost shadow artifact. In: Proceedings of the 1st ACM workshop on multimedia in forensics,(mifor09), pp 49–54Google Scholar
  41. 41.
    Zhu S, Ma KK (2000) A new diamond search algorithm for fast block-matching motion estimation. IEEE Trans Image Process 9(2):287–290CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.College of Computer Science and Electronic EngineeringHunan UniversityChangshaChina
  2. 2.School of Information Science and EngineeringJishou UniversityJishouChina

Personalised recommendations