Skip to main content
SpringerLink
Log in

A dynamic causal topic model for mining activities from complex videos

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, a novel probabilistic topic model is proposed for mining activities from complex video surveillance scenes. In order to handle the temporal nature of the video data, we devise a dynamical causal topic model (DCTM) that can detect the latent topics and causal interactions between them. The model is based on the assumption that all temporal relationships between latent topics at neighboring time steps follow a noisy-OR distribution. And the parameter of the noisy-OR distribution is estimated by a data driven approach based on the idea of nonparametric Granger causality statistic. Furthermore, for convergence analysis during model learning process, the Kullback-Leibler between the prior and the posterior distributions is calculated. At last, using the causality matrix learned by DCTM, the total causal influence of each topic is measured. We evaluate the proposed model through experimentations on several challenging datasets and demonstrate that our model can identify the high influence activity in crowded scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Blei DM, Lafferty JD (2006) Dynamic topic models[C] Proceedings of the 23rd international conference on machine learning. ACM, pp 113–120

    Google Scholar 

  2. Blei DM, Ng AY, Jordan MI (2003) Latent dirichlet allocation[J]. J Mach Learn Res 3:993–1022

    MATH  Google Scholar 

  3. Chua FCT, Oentaryo RJ, Lim EP (2015) Using linear dynamical topic model for inferring temporal social correlation in latent Space[J]. Computer Science

  4. Emonet R, Varadarajan J, Odobez J (2011) Extracting and locating temporal motifs in video scenes using a hierarchical non parametric Bayesian model[C]. IEEE Computer Vision and Pattern Recognition 3233–3240

  5. Fan Y, Yang H, Zheng S et al (2013) Video sensor-based complex scene analysis with Granger causality[J]. Sensors 13(10):13685–13707

    Article  Google Scholar 

  6. Faruquie TA, Kalra PK, Banerjee S (2009) Time based activity inference using latent dirichlet allocation[C]. BMVC 1–10

  7. Gu B, Sheng VS (2016) A robust regularization path algorithm for v-support vector classification[J]. IEEE Transactions on Neural Networks and Learning Systems 28(5):1241-1248

  8. Gu B, Sun X, Sheng VS (2016) Structural minimax probability machine. IEEE Transactions on Neural Networks and Learning Systems. doi:10.1109/TNNLS.2016.2544779

  9. Gu B, Sheng VS, Tay KY et al (2015) Incremental support vector learning for ordinal regression[J]. IEEE Transactions on Neural Networks and Learning Systems 26(7):1403–1416

    Article  MathSciNet  Google Scholar 

  10. Gu B, Sheng VS, Wang Z et al (2015) Incremental learning for v-support vector regression[J]. Neural Netw 67:140–150

    Article  Google Scholar 

  11. Hofmann T (2001) Unsupervised learning by probabilistic latent semantic analysis[J]. Mach Learn 42(1-2):177–196

    Article  MATH  Google Scholar 

  12. Hospedales T, Gong S, Xiang T (2009) Markov clustering topic model for mining behaviour in video[C] IEEE international conference on computer vision, IEEE, pp 1165–1172

    Google Scholar 

  13. Kinoshita A, Takasu A, Adachi J (2015) Real-time traffic incident detection using a probabilistic topic model[J]. Inf Syst 54(C):169–188

    Article  Google Scholar 

  14. Kuettel D, Breitenstein MD, Gool LV et al (2010) What’s going on? Discovering spatio-temporal dependencies in dynamic scenes[C]. IEEE Computer Vision and Pattern Recognition 1951–1958

  15. Kular D, Ribeiro E (2015) Analyzing activities in videos using latent dirichlet allocation and granger causality[C] International symposium on visual computing. Springer International Publishing, pp 647–656

    Google Scholar 

  16. Li J, Gong S, Xiang T (2008) Global behaviour inference using probabilistic latent semantic analysis[C]. BMVC 3231:3232

    Google Scholar 

  17. Li Y, Lu H, Li J et al (2016) Underwater image de-scattering and classification by deep neural network[J]. Comput Electr Eng 54:68–77

    Article  Google Scholar 

  18. Lu H, Li Y, Nakashima S et al (2016) Single image dehazing through improved atmospheric light estimation[J]. Multimedia Tools and Applications 75(24):17081–17096

    Article  Google Scholar 

  19. Mccaffery JP, Maida AS (2013) Toward a causal topic model for video scene analysis[C] International joint conference on neural networks, pp 1–8

    Google Scholar 

  20. Nedungadi A, Rangarajan G, Jain N, Ding M (2008) Analyzing multiple spike trains with nonparametric granger causality. J Comput Neurosci 27:55–64

    Article  MathSciNet  Google Scholar 

  21. Qian S, Zhang T, Xu C et al (2016) Multi-modal event topic model for social event analysis[J]. IEEE Trans Multimedia 18(2):233–246

    Article  Google Scholar 

  22. Song L, Jiang F, Shi Z et al (2011) Understanding dynamic scenes by hierarchical motion pattern mining[C] IEEE international conference on multimedia and expo (ICME), 2011. IEEE, pp 1–6

    Google Scholar 

  23. Teh YW, Jordan MI, Beal MJ, Blei DM (2006) Hierarchical dirichlet processes. J Am Stat Assoc 101(476)

  24. Varadarajan J, Emonet R, Odobez JM (2013) A sequential topic model for mining recurrent activities from long term video Logs[J]. Int J Comput Vis 103 (1):100–126

    Article  MathSciNet  MATH  Google Scholar 

  25. Wang X, Ma X, Grimson WEL (2009) Unsupervised activity perception in crowded and complicated scenes using hierarchical bayesian models[J]. IEEE Trans Pattern Anal Mach Intell 31(3):539–555

    Article  Google Scholar 

  26. Xue J, Eguchi K (2016) Sequential correspondence hierarchical dirichlet processes for video data analysis[C]. ACM ACM 229–233

  27. Zach C, Pock T, Bischof H (2007) A duality based approach for realtime TV-l, 1 optical Flow[J]. Lect Notes Comput Sci 4713(5):214–223

    Article  Google Scholar 

  28. Zhou L, Hu RQ, Qian Y et al (2013) Energy-spectrum efficiency tradeoff for video streaming over mobile ad hoc networks[J]. IEEE J Sel Areas Commun 31 (5):981–991

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the associated editor and all the anonymous reviewers for their valuable comments and suggestions. This work was partly supported by Natural Science Foundation of Jiangsu Province(Grant No. BK20160908), and the Key Lab of Broadband Wireless Communication and Sensor Network Technology(Grant No.NYKL2015012), and the National Natural Science Foundation of China (Grant No. 61401228, 61501253) and the Basic Research Program of Jiangsu Province(Natural Science Foundation)(Grant No. BK20151506), and the China Postdoctoral Science Foundation(Grant No. 2015M581841), and the Postdoctoral Science Foundation of Jiangsu Province (Grant No. 1501019A), and the Priority Academic Program Development of Jiangsu Higer Education Institutions(PAPD), and the Jiangsu Collaborative Innovation Center on Atmospheric Environment and Equipment Technology(CICAEET), and the Nanjing University of Information Science and Technology Research Foundation for Talented Scholars (Grant No. 2015r014).

Author information

Authors and Affiliations

  1. Key Lab of Ministry of Education for Broad Band Wireless Communication and Sensor Network Technology, Nanjing University of Posts and Telecommunications, Nanjing, China

    Yawen Fan, Quan Zhou, Wenjing Yue & Weiping Zhu

  2. School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, China

    Quan Zhou

  3. Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, Canada

    Weiping Zhu

Authors
  1. Yawen Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Quan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenjing Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weiping Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yawen Fan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, Y., Zhou, Q., Yue, W. et al. A dynamic causal topic model for mining activities from complex videos. Multimed Tools Appl 77, 10669–10684 (2018). https://doi.org/10.1007/s11042-017-4760-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4760-4

Keywords

Search

Navigation