Skip to main content
SpringerLink
Log in

Classification-Driven Video Analytics for Critical Infrastructure Protection

  • Chapter
  • First Online:
Recent Advances in Computational Intelligence in Defense and Security

Part of the book series: Studies in Computational Intelligence ((SCI,volume 621))

Abstract

At critical infrastructure sites, either large number of onsite personnel, or many cameras are needed to keep all key access points under continuous observation. With the proliferation of inexpensive high quality video imaging devices, and improving internet bandwidth, the deployment of large numbers of cameras monitored from a central location have become a practical solution. Monitoring a high number of critical infrastructure sites may cause the operator of the surveillance system to become distracted from the many video feeds, possibly missing key events, such as suspicious individuals approaching a door or leaving an object behind. An automated monitoring system for these types of events within a video feed alleviates some of the burden placed on the operator, thereby increasing the overall reliability and performance of the system, as well as providing archival capability for future investigations. In this work, a solution that uses a background subtraction-based segmentation method to determine objects within the scene is proposed. An artificial neural network classifier is then employed to determine the class of each object detected in every frame. This classification is then temporally filtered using Bayesian inference in order to minimize the effect of occasional misclassifications. Based on the object’s classification and spatio-temporal properties, the behavior is then determined. If the object is considered of interest, feedback is provided to the background subtraction segmentation technique for background fading prevention reasons. Furthermore any undesirable behavior will generate an alert, to spur operator action.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Weiss, L.G.: Autonomous robots in the fog of war. IEEE Spectr. 48(8), 30–34, 56–57 (2011)

    Google Scholar 

  2. Amato, A., Di Lecce, V., Piuri, V.: Semantic Analysis and Understanding of Human Behavior in Video Streaming. Springer, New York (2013)

    Book  Google Scholar 

  3. OpenCV dev team: OpenCV 3.0.0-dev documentation. WWW: http://docs.opencv.org/master/index.html, (2014). Accessed June 2014

  4. Anthony, G., Gregg, H., Tshilidzi, M.: Image classification using SVMs: one-against-one versus one-against-all. In: Proceedings of the Asian Conference on Remote Sensing (2007)

    Google Scholar 

  5. Banerjee, B., Bhattacharjee, T., Chowdhury, N.: Image object classification using scale invariant feature transform descriptor with support vector machine classifier with histogram intersection kernel. In: Information and Communication Technologies: Communications in Computer and Information Science, pp. 443–448. Springer, Berlin (2010)

    Google Scholar 

  6. Cramer, K., Singer, Y.: On the algorithmic implementation of multiclass kernel-based vector machines. J. Mach. Learn. Res. 2, 254–292 (2001)

    Google Scholar 

  7. Shotton, J., Winn, J., Rother, C., Criminisi, A.: TextonBoost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int. J. Comput. Vis. 81(1), 2–23 (2009)

    Article  Google Scholar 

  8. Lienhart, R., Kuranov, A., Pisarevsky, V.: Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection. Intel Corporation, Santa Clara (2002)

    Google Scholar 

  9. Voila, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the Converence on Computer Vision and Pattern Recognition, pp. 511–518 (2001)

    Google Scholar 

  10. Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: International Conference on Computer Vision Theory and Applications (VISAPP’09) (2009)

    Google Scholar 

  11. Muja, M., Lowe, D.G.: Scalable nearest neighbor algorithms for high dimensional data. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), vol. 36 (2014)

    Google Scholar 

  12. Zhang, G.P.: Neural networks for classification: a survey. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 30(4), 451–462 (2000)

    Google Scholar 

  13. Goerick, C., Noll, D., Werner, M.: Artificial neural networks in real time car detection and tracking applications. Pattern Recogn. Lett. Neural Netw. Comput. Vis. Appl. 17(4), 335–343 (1996)

    Article  Google Scholar 

  14. Haykin, S.: Neural Neworks: A Comprehensive Foundation, 2nd edn. Prentice-Hall Inc., Upper-Saddle River (1999)

    MATH  Google Scholar 

  15. Roerdink, J.B., Meijster, A.: The watershed transform: definitions, algorithms and parallelization strategies. Fundamenta Informaticae 41, 187–228 (2001)

    MathSciNet  MATH  Google Scholar 

  16. Alsabti, K., Ranka, S., Singh, V.: An Efficient k-means Clustering Algorithm. Syracuse University SURFACE Electrical Engineering and Computer Science, Syracuse (1997)

    Google Scholar 

  17. Ding, C., He, X.: K-means clustering via principal component analysis. In: Proceedings of the International Conference on Machine Learning, Banff (2004)

    Google Scholar 

  18. Ryan, K., Amer, A., Gagnon, L.: Video object segmentation based on object enhancement and region merging. In: IEEE Internation Conference on Multimedia and Expo, Toronto (2006)

    Google Scholar 

  19. EL Hassani, M., Jehan-Besson, S., Brun, L., Revenu, M., Duranton, M., Tschumperlé, D., Rivasseau, D.: A time-consistent video segmentation algorithm designed for real-time implementation. In: VLSI Design, vol. 2008, p. 12 (2008)

    Google Scholar 

  20. Hu, W.-C.: Real-time on-line video object segmentation based on motion detection without background construction. Int. J. Innov. Comput. Inf. Control 7(4), 1845–1860 (2011)

    Google Scholar 

  21. Rother, C., Kolmogorov, V., Blake, A.: GrabCut: Interactive foreground extraction using iterated graph cuts. ACM Trans. Graphics (SIGGRAPH) 23, 309–314 (2004)

    Article  Google Scholar 

  22. Bradski, G.R.: Computer Vision Face Tracking for Use in a Perceptual User Interface. Intel Corporation (1998)

    Google Scholar 

  23. Babenko, B., Yang, M.-H., Belongie, S.: Visual tracking with online multiple instance learning. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), August (2011)

    Google Scholar 

  24. Lowe, D.G.: Object recognition from local scale-invariant features. Proc. Int. Conf. Comput. Vis. 2, 1150–1157 (1999)

    Google Scholar 

  25. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. Comput. Vis. Image Underst. (CVIU) 110(3), 346–359 (2008)

    Article  Google Scholar 

  26. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: and efficient alternative to SIFT or SURF. In: IEEE Internation Conference on Computer Vision (ICCV) (2011)

    Google Scholar 

  27. Csurka, G., Dance, C.R., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: ECCV International Workshop on Statistical Learning in Computer Vision (2004)

    Google Scholar 

  28. Friedman, N., Russell, S.: Image segmentation in video sequences: a probabilistic approach. In: Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI’97) (1997)

    Google Scholar 

  29. KaewTraKulPong, P., Bowden, R.: An improved adaptive background mixture model for real-time tracking with shadow detection. In: Proceedings of the European Workshop on Advanced Video Based Surveillance Systems (2001)

    Google Scholar 

  30. Zivkovic, Z.: Improved adaptive Gaussian mixture model for background subtraction. In: Proceedings of the International Conference of Pattern Recognition (2004)

    Google Scholar 

  31. Sheikh, Y., Shah, M.: Bayesian modeling of dynamic scenes for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 27(11), 1778–1792 (2005)

    Article  Google Scholar 

  32. Avgerinakis, K., Briassouli, A., Kompatsiaris, I.: Real Time Illumination Invariant Motion Change Detection. In: ACM-MM ARTEMIS International Workshop. Firenze, Italy (2010)

    Book  Google Scholar 

  33. Drew, M.S., Wei, J., Li, Z.-N.: Illumination-invariant color object recognition via compressed chromaticity histograms of color-channel-normalized images. In: IEEE International Conference on Computer Vision, Bombay, India (1998)

    Google Scholar 

  34. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)

    Article  Google Scholar 

  35. The PASCAL Visual Object Classes Homepage. WWW: http://pascallin.ecs.soton.ac.uk/challenges/VOC/ (2014). Accessed 7 July 2014

  36. Banerjee, B., Bhattacharjee, T., Chowdhury, N.: Image object classification using scale invariant feature transform descriptor with support vector machine classifier with histogram intersection kernel. In: Information and Communication Technologies: Communications in Computer and Information Science, pp. 443–448. Springer, Berlin (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research & Engineering Division, Larus Technologies, Ottawa, Canada

    Phillip Curtis, Moufid Harb & Rami Abielmona

  2. School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada

    Phillip Curtis, Rami Abielmona & Emil Petriu

Authors
  1. Phillip Curtis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Moufid Harb
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rami Abielmona
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Emil Petriu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Phillip Curtis .

Editor information

Editors and Affiliations

  1. Larus Technologies Corporation, Ottawa, Ontario, Canada

    Rami Abielmona

  2. Larus Technologies Corporation, Ottawa, Ontario, Canada

    Rafael Falcon

  3. Faculty of Computer Science, Dalhousie University, Halifax, Nova Scotia, Canada

    Nur Zincir-Heywood

  4. School of Engineering and Information Technology, University of New South Wales, Canberra, Aust Capital Terr, Australia

    Hussein A. Abbass

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Curtis, P., Harb, M., Abielmona, R., Petriu, E. (2016). Classification-Driven Video Analytics for Critical Infrastructure Protection. In: Abielmona, R., Falcon, R., Zincir-Heywood, N., Abbass, H. (eds) Recent Advances in Computational Intelligence in Defense and Security. Studies in Computational Intelligence, vol 621. Springer, Cham. https://doi.org/10.1007/978-3-319-26450-9_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-26450-9_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-26448-6

  • Online ISBN: 978-3-319-26450-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation