Advertisement

A novel automatic shot boundary detection algorithm: robust to illumination and motion effect

  • Alok SinghEmail author
  • Dalton Meitei Thounaojam
  • Saptarshi Chakraborty
Original Paper
  • 70 Downloads

Abstract

Many researches have been done on shot boundary detection, but the performance of shot boundary detection approaches is yet to be addressed for the videos having sudden illumination and object/camera motion effects efficiently. In this paper, a novel dual-stage approach for an abrupt transition detection is proposed which is able to withstand under certain illumination and motion effects. Firstly, an adaptive Wiener filter is applied to the lightness component of the frame to retain some important information on both frequencies and LBP-HF is extracted to reduce the illumination effect. From the experimentation, it is also confirmed that the motion effect is also reduced in the first stage. Secondly, Canny edge difference is used to further remove the illumination and motion effects which are not handled in the first stage. TRECVid 2001 and TRECVid 2007 datasets are applied to analyze and validate our proposed algorithm. Experimental results manifest that the proposed system outperforms the state-of-the-art shot boundary detection techniques.

Keywords

LBP-HF Shot boundary detection Abrupt Adaptive threshold 

Notes

References

  1. 1.
    Abdulhussain, S.H., Ramli, A.R., Saripan, M.I., Mahmmod, B.M., Al-Haddad, S.A.R., Jassim, W.A.: Methods and challenges in shot boundary detection: a review. Entropy 20(4), 214 (2018)CrossRefGoogle Scholar
  2. 2.
    Ahonen, T., Matas, J., He, C., Pietikäinen, M.: Rotation invariant image description with local binary pattern histogram fourier features. In: Salberg, A.B., Hardeberg, J.Y., Jenssen, R. (eds.) Image Analysis, pp. 61–70. Springer, Berlin (2009)CrossRefGoogle Scholar
  3. 3.
    Chakraborty, S., Thounaojam, D.M.: A novel shot boundary detection system using hybrid optimization technique. Appl. Intell. (2019).  https://doi.org/10.1007/s10489-019-01444-1 CrossRefGoogle Scholar
  4. 4.
    Chen, L.H., Hsu, B.C., Su, C.W.: A supervised learning approach to flashlight detection. Cybern. Syst. 48(1), 1–12 (2017)CrossRefGoogle Scholar
  5. 5.
    Domnic, S.: Walsh–Hadamard transform kernel-based feature vector for shot boundary detection. IEEE Trans. Image Process. 23(12), 5187–5197 (2014)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Fu, Q., Zhang, Y., Xu, L., Li, H.: A method of shot-boundary detection based on HSV space. In: Ninth International Conference on Computational Intelligence and Security, pp. 219–223 (2013)Google Scholar
  7. 7.
    Hassanien, A., Elgharib, M.A., Selim, A., Hefeeda, M., Matusik, W.: Large-scale, fast and accurate shot boundary detection through spatio-temporal convolutional neural networks. CoRR arXiv:1705.03281 (2017)
  8. 8.
    Heng, W.J., Ngan, K.N.: The implementation of object-based shot boundary detection using edge tracing and tracking. IEEE Int. Symp. Circuits Syst. VLSI 4, 439–442 (1999)Google Scholar
  9. 9.
    Heng, W.J., Ngan, K.N.: An object-based shot boundary detection using edge tracing and tracking. J. Vis. Commun. Image Represent. 12(3), 217–239 (2001)CrossRefGoogle Scholar
  10. 10.
    Huan, Z., Xiuhuan, L., Lilei, Y.: Shot boundary detection based on mutual information and canny edge detector. Int. Conf. Comput. Sci. Softw. Eng. 2, 1124–1128 (2008)Google Scholar
  11. 11.
    Kaabneh, K., Alia, O., Suleiman, A., Abuirbaleh, A.: Video segmentation via dual shot boundary detection (DSBD). In: International Conference on Information and Communication Technologies, vol. 1. IEEE, pp. 1530–1533 (2006)Google Scholar
  12. 12.
    Kanungo, P., Kar, T.: Cut detection using block based center symmetric local binary pattern. In: International Conference on Man and Machine Interfacing, pp. 1–5 (2015)Google Scholar
  13. 13.
    Kar, T., Kanungo, P.: A texture based method for scene change detection. In: International Conference on Power, Communication and Information Technology Conference, pp. 72–77 (2015)Google Scholar
  14. 14.
    Kar, T., Kanungo, P.: A motion and illumination resilient framework for automatic shot boundary detection. Signal Image Video Process. 11(7), 1237–1244 (2017)CrossRefGoogle Scholar
  15. 15.
    Lan, X., Zhang, S., Yuen, P.C., Chellappa, R.: Learning common and feature-specific patterns: a novel multiple-sparse-representation-based tracker. IEEE Trans. Image Process. 27(4), 2022–2037 (2018)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Lan, X., Ye, M., Shao, R., Zhong, B., Jain, D.K., Zhou, H.: Online non-negative multi-modality feature template learning for RGB-assisted infrared tracking. IEEE Access 7, 67761–67771 (2019)CrossRefGoogle Scholar
  17. 17.
    Lan, X., Ye, M., Shao, R., Zhong, B., Yuen, P.C., Zhou, H.: Learning modality-consistency feature templates: a robust RGB-infrared tracking system. IEEE Trans. Ind. Electron. (2019).  https://doi.org/10.1109/TIE.2019.2898618 CrossRefGoogle Scholar
  18. 18.
    Li, Y., Lu, Z., Niu, X.: Fast video shot boundary detection framework employing pre-processing techniques. IET Image Process. 3(3), 121–134 (2009)CrossRefGoogle Scholar
  19. 19.
    Lim, J.S.: Two-Dimensional Signal and Image Processing. Prentice Hall, Englewood Cliffs, NJ (1990)Google Scholar
  20. 20.
    Liu, T., Chan, S.: Automatic shot boundary detection algorithm using structure–aware histogram metric. In: International Conference on Digital Signal Processing, pp. 541–546 (2014)Google Scholar
  21. 21.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)CrossRefGoogle Scholar
  22. 22.
    Rashmi, B.S., Nagendraswamy, H.S.: Video shot boundary detection using midrange local binary pattern. In: International Conference on Advances in Computing, Communications and Informatics, pp. 201–206 (2016)Google Scholar
  23. 23.
    Srilakshmi, B., Sandeep, R.: Shot boundary detection using structural similarity index. In: Fifth International Conference on Advances in Computing and Communications (ICACC), pp. 439–442 (2015)Google Scholar
  24. 24.
    Tang, S., Feng, L., Kuang, Z., Chen, Y., Zhang, W.: Fast video shot transition localization with deep structured models. CoRR arXiv:1808.04234 (2018)
  25. 25.
    Thounaojam, D.M., Khelchandra, T., Singh, K.M., Roy, S.: A genetic algorithm and fuzzy logic approach for video shot boundary detection. Comput. Intell. Neurosci. 2016, 14 (2016)CrossRefGoogle Scholar
  26. 26.
    Tong, W., Song, L., Yang, X., Qu, H., Xie, R.: CNN-based shot boundary detection and video annotation. In: IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, pp. 1–5 (2015)Google Scholar
  27. 27.
    Waghmare, M.S.P., Bhide, A.: Shot boundary detection using histogram differences. Int. J. Adv. Res. Electron. Commun. Eng. 3, 1460–1464 (2014)Google Scholar
  28. 28.
    Warhade, K.K., Merchant, S.N., Desai, U.B.: Avoiding false positive due to flashlights in shot detection using illumination suppression algorithm. In: International Conference on Visual Information Engineering, pp. 377–381 (2008)Google Scholar
  29. 29.
    Warhade, K.K., Merchant, S.N., Desai, U.B.: Shot boundary detection in the presence of fire flicker and explosion using stationary wavelet transform. Signal Image Video Process. 5(4), 507–515 (2011)CrossRefGoogle Scholar
  30. 30.
    Warhade, K.K., Merchant, S.N., Desai, U.B.: Shot boundary detection in the presence of illumination and motion. Signal Image Video Process. 7(3), 581–592 (2013)CrossRefGoogle Scholar
  31. 31.
    Xie, X., Zheng, W.S., Lai, J., Yuen, P.C.: Face illumination normalization on large and small scale features. In: Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)Google Scholar
  32. 32.
    Xu, J., Song, L., Xie, R.: Shot boundary detection using convolutional neural networks. In: Visual Communications and Image Processing, pp. 1–4 (2016)Google Scholar
  33. 33.
    Zou, X., Kittler, J., Messer, K.: Illumination invariant face recognition: a survey. In: First IEEE International Conference on Biometrics: Theory, Applications, and Systems, pp. 1–8 (2007)Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Computer Vision Lab, Department of Computer Science and EngineeringNational Institute of TechnologySilcharIndia

Personalised recommendations