Multimedia Tools and Applications

, Volume 78, Issue 14, pp 20361–20382 | Cite as

Shot boundary detection based on orthogonal polynomial

  • Sadiq H. AbdulhussainEmail author
  • Abd Rahman Ramli
  • Basheera M. Mahmmod
  • M. Iqbal Saripan
  • S. A. R. Al-Haddad
  • Wissam A. Jassim


Shot boundary detection (SBD) is a substantial step in video content analysis, indexing, retrieval, and summarization. SBD is the process of automatically partitioning video into its basic units, known as shots, through detecting transitions between shots. The design of SBD algorithms developed from simple feature comparison to rigorous probabilistic and using of complex models. Nevertheless, accelerate the detection of transitions with higher accuracy need to be improved. Extensive research has employed orthogonal polynomial (OP) and their moments in computer vision and signal processing owing to their powerful performance in analyzing signals. A new SBD algorithm based on OP has been proposed in this paper. The Features are derived from orthogonal transform domain (moments) to detect the hard transitions in video sequences. Moments are used because of their ability to represent signal (video frame) without information redundancy. These features are the moments of smoothed and gradients of video frames. The moments are computed using a developed OP which is squared Krawtchouk-Tchebichef polynomial. These moments (smoothed and gradients) are fused to form a feature vector. Finally, the support vector machine is utilized to detect hard transitions. In addition, a comparison between the proposed algorithm and other state-of-the-art algorithms is performed to reinforce the capability of the proposed work. The proposed algorithm is examined using three well-known datasets which are TRECVID2005, TRECVID2006, and TRECVID2007. The outcomes of the comparative analysis show the superior performance of the proposed algorithm against other existing algorithms.


Shot boundary detection Temporal video segmentation Hard transition Abrupt transition Orthogonal polynomial Orthogonal moments 



We would like express our heartfelt thanks to all of the anonymous reviewers for their efforts, valuable comments and suggestions.

Compliance with Ethical Standards

Conflict of interests

The authors declare that they have no conflict of interest.


  1. 1.
    Abdulhussain SH (2017) On computational aspects of Tchebichef polynomials for higher polynomial order. IEEE Access 5(1):2470–2478Google Scholar
  2. 2.
    Abdulhussain SH, Ramli AR, Mahmmod BM, Al-Haddad SAR, Jassim WA (2017) Image edge detection operators based on orthogonal polynomials. Int J Image Data Fus 8(3):293–308Google Scholar
  3. 3.
    Abdulhussain SH, Ramli AR, Al-Haddad SAR, Mahmmod BM, Jassim WA (2018) Fast recursive computation of Krawtchouk polynomials. J Math Imaging Vis 60(3):285–303MathSciNetzbMATHGoogle Scholar
  4. 4.
    Abdulhussain SH, Ramli AR, Saripan MI, Mahmmod BM, Al-Haddad S, Jassim WA (2018) Methods and challenges in shot boundary detection: a review. Entropy 20(4):214Google Scholar
  5. 5.
    Birinci M, Kiranyaz S (2014) A perceptual scheme for fully automatic video shot boundary detection. Signal Process Image Commun 29(3):410–423Google Scholar
  6. 6.
    Camara-Chavez G, Precioso F, Cord M, Phillip-Foliguet S, de A Araujo A (2007) Shot boundary detection by a hierarchical supervised approach. In: 2007 14th international workshop on systems, signals and image processing and 6th EURASIP conference focused on speech and image processing, multimedia communications and services, pp 197–200Google Scholar
  7. 7.
    Chang C-C, Lin C-J (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27Google Scholar
  8. 8.
    Chaves GC (2007) Video content analysis by active learning. PhD Thesis, Federal University of Minas GeraisGoogle Scholar
  9. 9.
    Chen X, He F, Yu H (2018) A matting method based on full feature coverage. Multimed Tools Appl, 1–29Google Scholar
  10. 10.
    Cooper M, Foote J, Adcock J, Casi S (2003) Shot boundary detection via similarity analysis. In: Proceedings of the TRECVID workshopGoogle Scholar
  11. 11.
    Gargi U, Kasturi R, Strayer SH (2000) Performance characterization of video-shot-change detection methods. IEEE Trans Circ Syst 8(10):4761–4766Google Scholar
  12. 12.
    Hsu C-W, Chang C-C, Lin C-J et al (2003) A practical guide to support vector classification. Department of Computer Science and Information EngineeringGoogle Scholar
  13. 13.
    Hu W, Xie N, Li L, Zeng X, Maybank S (2011) A survey on visual content-based video indexing and retrieval. IEEE Trans Syst Man Cybern Part C (Appl Rev) 41(6):797–819Google Scholar
  14. 14.
    Jaffrė G, Joly P, Haidar S (2004) The Samova shot boundary detection for TRECVID evaluation 2004Google Scholar
  15. 15.
    Janwe NJ, Bhoyar KK (2013) Video shot boundary detection based on JND color histogram. In: 2013 IEEE Second international conference on image information processing (ICIIP), pp 476–480Google Scholar
  16. 16.
    Ji QG, Feng JW, Zhao J, Lu ZM (2010) Effective dissolve detection based on accumulating histogram difference and the support point. In: 2010 First international conference on pervasive computing signal processing and applications (PCSPA), pp 273–276Google Scholar
  17. 17.
    Kar T, Kanungo P (2017) A motion and illumination resilient framework for automatic shot boundary detection. Signal Image Vid Process 11(7):1237–1244Google Scholar
  18. 18.
    Koprinska I, Carrato S (2001) Temporal video segmentation: a survey. Signal Process Image Commun 16(5):477–500Google Scholar
  19. 19.
    Krulikovska L, Pavlovic J, Polec J, Cernekova Z (2010) Abrupt cut detection based on mutual information and motion prediction. In: ELMAR 2010 proceedings, pp 89–92Google Scholar
  20. 20.
    Küçüktunç O, Güdükbay U, Ulusoy Ö (2010) Fuzzy color histogram-based video segmentation. Comput Vis Image Underst 114(1):125–134Google Scholar
  21. 21.
    Li K, He F, Yu H, Chen X (2017) A parallel and robust object tracking approach synthesizing adaptive Bayesian learning and improved incremental subspace learning. Front Comput Sci, 1–20Google Scholar
  22. 22.
    Li K, He F-Z, Yu H-P (2018) Robust visual tracking based on convolutional features with illumination and occlusion handing. J Comput Sci Technol 33(1):223–236Google Scholar
  23. 23.
    Liu C, Wang D, Zhu J, Zhang B (2013) Learning a contextual multi-thread model for Movie/TV scene segmentation. IEEE Trans Multimed 15(4):884–897Google Scholar
  24. 24.
    Liu L, Hua Y, Zhao Q, Huang H, Bovik AC (2016) Blind image quality assessment by relative gradient statistics and adaboosting neural network. Signal Process Image Commun 40(Supplement C):1–15Google Scholar
  25. 25.
    Mahmmod BM, Ramli AR, Abdulhussain SH, Al-Haddad SAR, Jassim WA (2017) Low-distortion MMSE speech enhancement estimator based on Laplacian prior. IEEE Access 5(1):9866–9881Google Scholar
  26. 26.
    Mahmmod BM, bin Ramli AR, Abdulhussain SH, Al-Haddad SAR, Jassim WA (2018) Signal compression and enhancement using a new orthogonal-polynomial-based discrete transform. IET Signal Process 12(1):129–142Google Scholar
  27. 27.
    Mondal J, Kundu MK, Das S, Chowdhury M (2017) Video shot boundary detection using multiscale geometric analysis of nsct and least squares support vector machine. Multimed Tools Appl, 1–23Google Scholar
  28. 28.
    Mukundan R (2004) Some computational aspects of discrete orthonormal moments. IEEE Trans Image Process 13(8):1055–1059MathSciNetGoogle Scholar
  29. 29.
    Noble WS (2006) What is a support vector machine? Nat Biotechnol 24(12):1565–1567Google Scholar
  30. 30.
    Over P, Ianeva T, Kraaij W, Smeaton AF, Val UD (2005) TRECVID 2005 - an overview. NIST, 1–27Google Scholar
  31. 31.
    Over P, Ianeva T, Kraaij W, Smeaton AF (2006) TRECVID 2006 - an overview. NIST, 1–29Google Scholar
  32. 32.
    Pacheco F, Cerrada M, Sänchez R-V, Cabrera D, Li C, de Oliveira JV (2017) Attribute clustering using rough set theory for feature selection in fault severity classification of rotating machinery. Expert Syst Appl 71:69–86Google Scholar
  33. 33.
    Parmar M, Angelides MC (2015) MAC-REALM: a video content feature extraction and modelling framework. Comput J 58(9):2135–2170Google Scholar
  34. 34.
    Porter SV, Mirmehdi M, Thomas BT (2000) Video cut detection using frequency domain correlation. In: 2000 Proceedings 15th international conference on pattern recognition, vol 3. IEEE, pp 409–412Google Scholar
  35. 35.
    Priya GL, Domnic S (2012) Edge strength extraction using orthogonal vectors for shot boundary detection. Procedia Technology 6:247–254Google Scholar
  36. 36.
    Priya LGG, Domnic S (2014) Walsh – Hadamard transform kernel-based feature vector for shot boundary detection. IEEE Trans Image Process 23(12):5187–5197MathSciNetzbMATHGoogle Scholar
  37. 37.
    Sasithradevi A, Roomi SMM, Raja R (2016) Non subsampled contourlet transform based shot boundary detection in videos. Int J Control Theory Appl 9(7):3231–3238Google Scholar
  38. 38.
    Sergios Theodoridis KK (2008) Pattern recognition, 4th edn. Academic PressGoogle Scholar
  39. 39.
    Smeaton AF, Over P, Doherty AR (2010) Video shot boundary detection: seven years of TRECVid activity. Comput Vis Image Underst 114(4):411–418Google Scholar
  40. 40.
    Solomon C, Breckon T (2011) Fundamentals of digital image processing: a practical approach with examples in matlab. Wiley-BlackwellGoogle Scholar
  41. 41.
    Swanberg D, Shu C-F, Jain RC (1993) Knowledge-guided parsing in video databases. In: IS&T/SPIE’s symposium on electronic imaging: science and technology. International Society for Optics and Photonics, pp 13–24Google Scholar
  42. 42.
    Thounaojam DM, Khelchandra T, Singh KM, Roy S (2016) A genetic algorithm and fuzzy logic approach for video shot boundary detection. Comput Intell Neurosci 2016(Article ID 8469428):14Google Scholar
  43. 43.
    Tong W, Song L, Yang X, Qu H, Xie R (2015) Ieee CNN-based shot boundary detection and video annotation. In: 2015 IEEE international symposium on broadband multimedia systems and broadcastingGoogle Scholar
  44. 44.
    Urhan O, Gullu MK, Erturk S (2006) Modified phase-correlation based robust hard-cut detection with application to archive film. IEEE Trans Circ Syst Vid Technol 16(6):753–770Google Scholar
  45. 45.
    Vlachos T (2000) Cut detection in video sequences using phase correlation. IEEE Signal Process Lett 7(7):173–175Google Scholar
  46. 46.
    Xu J, Tang YY, Zou B, Xu Z, Li L, Lu Y (2015) The generalization ability of online SVM classification based on Markov sampling. IEEE Trans Neural Netw Learn Syst 26(3):628–639MathSciNetGoogle Scholar
  47. 47.
    Yan X-H, He F-Z, Chen Y-L (2017) A novel hardware/software partitioning method based on position disturbed particle swarm optimization with invasive weed optimization. J Comput Sci Technol 32(2):340–355MathSciNetGoogle Scholar
  48. 48.
    Yan X, He F, Hou N, Ai H (2018) An efficient particle swarm optimization for large-scale hardware/software co-design system. Int J Coop Inf Syst 27(01):1741001Google Scholar
  49. 49.
    Yoo H-W, Ryoo H-J, Jang D-S (2006) Gradual shot boundary detection using localized edge blocks. Multimed Tools Appl 28(3):283–300Google Scholar
  50. 50.
    Youssef B, Fedwa E, Driss A, Ahmed S (2017) Shot boundary detection via adaptive low rank and svd-updating. Comput Vis Image Understand 161(Supplement C):20–28Google Scholar
  51. 51.
    Zabih R (1999) A feature-based algorithm for detecting and classifying production effects. Multimed Syst 7(2):119–128Google Scholar
  52. 52.
    Zabih R, Miller J, Mai K (1995) A feature-based algorithm for detecting and classifying scene breaks. In: Proceedings of the third ACM international conference on multimedia MULTIMEDIA 95, vol 95, pp 189–200Google Scholar
  53. 53.
    Zhang H, Kankanhalli A, Smoliar SW (1993) Automatic partitioning of full-motion video. Multimed Syst 1(1):10–28Google Scholar
  54. 54.
    Zhou Y, He F, Qiu Y (2017) Dynamic strategy based parallel ant colony optimization on gpus for tsps. Sci Chin Inf Sci 60(6):068102Google Scholar
  55. 55.
    Zhou Y, He F, Hou N, Qiu Y (2018) Parallel ant colony optimization on multi-core simd cpus. Futur Gener Comput Syst 79:473–487Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer EngineeringUniversity of BaghdadBaghdadIraq
  2. 2.Department of Computer and Communication System EngineeringUniversiti Putra MalaysiaSelangorMalaysia
  3. 3.ADAPT Center, School of Engineering, Trinity College DublinUniversity of DublinDublin 2Ireland

Personalised recommendations