Digital videos are an incredibly important source of information, and as evidence, they are highly inculpatory. Digital videos are also inherently prone to conscious semantic manipulations, such as copy–paste forgeries, which involve insertion or removal of objects into or from a set of frames. Such forgeries involve direct manipulation of the information presented by a video scene, thus having an immediate effect on the meaning conveyed by that scene. Given the highly influential nature of video data and the fact that they are easy to manipulate, it becomes important to devise measures that can help ascertain their integrity and authenticity, so that we can be certain of their ability to serve as reliable evidence. The challenge of detecting copy–paste forgeries in digital videos has been at the receiving end of much innovation over the last decade, and as a result, the available literature in this domain has grown to considerable proportions. However, thorough analysis of this literature appears to show that the task of detecting such forgeries necessitates the use of elaborate and operationally restrictive procedures, and somehow cannot be accomplished via a relatively simpler process, whose method of operation imposes little to no restrictions on its scope of applicability. With the aim of quashing this notion, in this paper, we present two simple forensic solutions that can enable an analyst to detect copy–paste forgeries quickly and effectively, without having to resort to any complicated analyses or relying on unrealistic presumptions. These solutions are based on optical flow inconsistency analysis and pattern noise abnormality analysis, and have been validated on a substantial set of realistically tampered test videos in a diverse experimental set-up, which is representative of a neutral testing platform and simulates a real-world heterogeneous forensic environment, where the analyst has no control over any of the variable parameters of the video creation or manipulation process. When tested in such an experimental set-up, the proposed solutions achieved an average accuracy rate of 98% and demonstrated attributes desired of an efficacious and practical forensic solution, all the while validating our initial hypothesis that not only can the task of copy–paste detection be accomplished in a fast and uncomplicated manner, but also that in an actual forgery scenario, the less onerous a forensic solution is, the more likely it is to succeed.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Technically, ‘tampering’ refers to the intentional modification of composition of something in a way that would render it harmful, whereas a ‘forgery’ refers to something that is falsely made with the intent to deceive. Albeit being subtly different, in this paper, as in the literature, these terms are used synonymously.
In the literature, the terms ‘SPN’ and ‘PRNU’ are often used synonymously, But, since the methods of their estimation are different, in this paper, we treat these two types of noise as distinct.
Please note that while there is a slight difference in the appearance of the menu bars in the original and tampered frames, the features of interest are offered by the portions of the television screens other than the menu bars; the bars—as such—are of no consequence to our analysis.
Singh, R.D., Aggarwal, N.: Video content authentication techniques: a comprehensive survey. Multimed. Syst. 24(2), 211–240 (2017)
Al-Sanjary, O.I., Ahmed, A.A., Sulonga, G.: Development of a video tampering dataset for forensic investigation. Forensic Sci. Int. 266, 565–572 (2016). https://www.youtube.com/channel/UCZuuu-iyZvPptbIUHT9tMrA
WangW., Farid, H.: Exposing digital forgeries in video by detecting duplication. In: 9th ACM Workshop on Multimedia and Security, pp. 35–42 (2007)
Bestagini, P., Milani, S., Tagliasacchi, M., Tubaro, S.: Local tampering detection in video sequences. In: 15th IEEE International Workshop on Multimedia Signal Processing, Pula, pp. 488–493 (2013)
Das, S., Darsan, G., Shreyas, L., Devan, D.: Blind detection method for video inpainting forgery. Int. J. Comput. Appl. 60(11), 33–37 (2012)
Lin, C.-S., Tsay, J.-J.: A passive approach for effective detection and localization of region-level video forgery with spatio–temporal coherence analysis. Dig. Investig. 11(2), 120–140 (2014)
D’Amiano,L., Cozzolino, D., Poggi, G., Verdoliva, L.: Video forgery detection and localization based on 3D patchmatch. In: IEEE International Conference on Multimedia and Expo Workshops, Turin, pp. 1–6 (2015)
Hsu, C.-C., Hung, T.-Y., Lin, C.-W., Hsu, C.-T.: Video forgery detection using correlation of noise residue. In: 10th IEEE Workshop on Multimedia Signal Processing, Cairns, pp. 170–174 (2008)
Chetty,G.: Blind and passive digital video tamper detection based on multimodal fusion. In: 14th WSEAS International Conference on Communications, Corfu, pp. 109–117 (2010)
Goodwin,J., Chetty, G.: Blind video tamper detection based on fusion of source features. In: IEEE International Conference on Digital Image Computing Techniques and Applications, Noosa, pp. 608–613 (2011)
Pandey,R.C., Singh, S.K., Shukla, K.K.: Passive copy-move forgery detection in videos. In: 5th IEEE International Conference on Computer and Communication Technology, Allahabad, pp. 301–306 (2014)
Labartino, D., Bianchi, T., De Rosa, A., Fontani, M., Vazquez-Padín, D., Piva, A., Barni, M.: Localization of forgeries in MPEG-2 video through GOP size and DQ analysis. In: 15th IEEE International Workshop on Multimedia Signal Processing, Pula, pp. 494–499 (2013)
Chen, S., Tan, S., Li, B., Huang, J.: Automatic detection of object–based forgery in advanced video. IEEE Trans. Circ. Syst. Video Technol. 26(11), 2138–2151 (2015)
Zhang, J., Su, Y., Zhang, M.: Exposing digital video forgery by ghost shadow artefact. In: 1st ACM Workshop on Multimedia in Forensics, Beijing, pp. 49–54 (2009)
Conotter, V., O’Brien, J.F., Farid, H.: Exposing digital forgeries in ballistic motion. IEEE Trans. Inf. Forensics Secur. 7(1), 283–296 (2012)
Richao, C., Gaobo, Y., Ningbo, Z.: Detection of object-based manipulation by the statistical features of object contour. Forensic Sci. Int. 236, 164–169 (2014)
Su, L., Huang, T., Yang, J.: A video forgery detection algorithm based on compressive sensing. Multimed. Tools Appl. 74(17), 6641–6656 (2015)
Li, L., Wang, X., Zhang, W., Yang, G., Hu, G.: Detecting removed object from video with stationary background. In: International Workshop on Digital Forensics and Watermarking, Shanghai, pp. 242–252 (2012)
Yao, Y., Shi, Y., Weng, S., Guan, B.: Deep learning for detection of object based forgery in advanced video. Symmetry 10(1), 3 (2017)
Bidokhti, A., Ghaemmaghami, S.: Detection of regional copy/move forgery in MPEG videos using optical flow. In: International Symposium on Artificial intelligence and Signal Processing, Mashhad, pp. 13–17 (2015)
Singh, R.D., Aggarwal, N.: Detection and localization of copy-paste forgeries in digital videos. Forensic Sci. Int. 281, 75–91 (2017)
Saddique, M., Asghar, K., Bajwa, U.I., Hussain, M., Habib, Z.: Video forgery detection and localization using texture analysis of consecutive frames. Adv. Electr. Comput. Eng. 19(3), 97–108 (2019)
Su, L., Luo, H., Wang, S.: A novel forgery detection algorithm for video foreground removal. IEEE Access 7, 109719–109728 (2019)
Kaur, K., Jindal, N.: Deep convolutional neural network for graphics forgery detection in video. Wirel. Pers. Commun. 112, 1763–1781 (2020)
Gibson,J.J.: The Perception of the Visual World. American Association for the Advancement of Science (1950)
Burton, A., Radford, J. (eds.): Thinking in Perspective: Critical Essays in the Study of Thought Processes. Methuen Publishing Ltd., London (1978)
Chao, J., Jiang, X., Sun, T.: A novel video inter–frame forgery model detection scheme based on optical flow consistency. Dig. Forensics Watermark. 7809, 267–281 (2013)
Zheng, L., Sun, T., Shi, Y.Q.: Inter–frame video forgery detection based on block-wise brightness variance descriptor. In: 13th International Workshop on Digital Forensics and Watermarking, Taipei, pp. 18–30 (2014)
Wang,W., Jiang, X., Wang, S., Meng, W.: Identifying video forgery process using optical flow. In: Digital Forensics and Watermarking, pp. 244–257 (2014)
Kingra, S., Aggarwal, N., Singh, R.D.: Inter–frame forgery detection using motion and brightness gradients. Multimed. Tools Appl. 76(4), 25767–25786 (2017)
De,A., Chadha, H., Gupta, S.: Detection of forgery in digital video. In: 10th World Multi Conference on Systems, Cybernetics and Informatics, pp. 229–233 (2006)
Wang, W., Farid, H.: Exposing digital forgeries in interlaced and deinterlaced video. IEEE Trans. Inf. Forensics Secur. 2(3), 438–449 (2007)
Mondaini,N., Caldelli, R., Piva, A., Barni, M., Cappellini, V.: Detection of malevolent changes in digital video for forensic applications. In: SPIE Conference on Security, Steganography and Watermarking of Multimedia Contents, vol 6505, no 1 (2007)
Kobayashi, M., Okabe, T., Sato, Y.: Detecting forgery from static-scene video based on inconsistency in noise level function. IEEE Trans. Inf. Forensics Secur. 5(4), 883–892 (2010)
Hyun, D.-K., Ryu, S.-J., Lee, H.-Y., Lee, H.-K.: Detection of upscale-crop and partial manipulation in surveillance video based on sensor pattern noise. Sensors 13(9), 12605–12631 (2013)
Singh, R.D., Aggarwal, N.: Detection of upscale-crop and splicing for digital video authentication. Dig. Investig. 21, 31–52 (2017)
Kumar, M., Rani, A., Srivastava, S.: Image forensics based on lighting estimation. Int. J. Image Graph. 19(3), 1950014 (2019)
Kumar, M., Srivastava, S.: Image authentication by assessing manipulations using illumination. Multimed. Tools Appl. 78, 12451–12463 (2019)
Kumar, M., Srivastava, S., Uddin, N.: Forgery detection using multiple light sources for synthetic images. Austral. J. Forensic Sci. 51(3), 243–250 (2019)
Aggarwal,A., Kumar, M.: Image surface texture analysis and classification using deep learning. Multimed. Tools Appl. (2020)
Singh, R.D., Aggarwal, N.: Optical flow and prediction residual based hybrid forensic system for inter-frame tampering detection. J. Circ. Syst. Comput. 26(7), 1750107-1-1750107–37 (2017)
Horn, B.K.P., Schunk, B.G.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)
Surrey University Library for Forensic Analysis (SULFA).: http://sulfa.cs.surrey.ac.uk/forged.php
Miss Ping (Tumba Ping Pong Show).: https://www.youtube.com/watch?v=5NO-fka_JTQ
Wiener, N.: Extrapolation, Interpolation, and Smoothing of Stationary Time Series. MIT Press, Cambridge (1964)
PU (Panjab University) Dataset.: http://pudataset.puchd.ac.in:8080/jspui/handle/123456789/22
Miss Pong (Tumba Ping Pong Show).: https://www.youtube.com/watch?v=dZZqaYgPrY0
FFmpeg Multimedia Framework.: https://www.ffmpeg.org/
Video Inpainting.: http://kedarpatwardhan.org/Research/VideoInpainting.html
Superhuman Tape Measure Skills.: https://www.youtube.com/watch?v=Wx_5GI0QRd
Magically throwing a ball through a mirror (OPTICAL ILLUSION).: https://www.youtube.com/watch?v=vKJhKb0ByoE
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Communicated by Y. Kong.
About this article
Cite this article
Singh, R.D., Aggarwal, N. Optical flow and pattern noise-based copy–paste detection in digital videos. Multimedia Systems (2021). https://doi.org/10.1007/s00530-020-00749-3
- Video content authentication
- Copy–paste detection
- Video forgery detection
- Tamper detection
- Optical flow
- Pattern noise
- Sensor pattern noise
- Photo response non-uniformity noise