Long-Range Spatio-Temporal Modeling of Video with Application to Fire Detection

  • Avinash Ravichandran
  • Stefano Soatto
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7573)


We describe a methodology for modeling backgrounds subject to significant variability over time-scales ranging from days to years, where the events of interest exhibit subtle variability relative to the normal mode. The motivating application is fire monitoring from remote stations, where illumination changes spanning the day and the season, meteorological phenomena resembling smoke, and the absence of sufficient training data for the two classes make out-of-the-box classification algorithms ineffective. We exploit low-level descriptors, incorporate explicit modeling of nuisance variability, and learn the residual normal-model variability. Our algorithm achieves state-of-the-art performance not only compared to other anomaly detection schemes, but also compared to human performance, both for untrained and trained operators.


Ground Truth Video Sequence Detection Performance Anomaly Detection Equal Error Rate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Liu, C.B., Ahuja, N.: Vision based fire detection. In: ICPR (2004)Google Scholar
  2. 2.
    Kolesov, I., Karasev, P., Tannenbaum, A., Haber, E.: Fire and smoke detection in video with optimal mass transport based optical flow and neural networks. In: ICIP (2010)Google Scholar
  3. 3.
    Habiboglu, Y., Gunay, O., Cetin, A.: Flame detection method in video using covariance descriptors. In: ICCASP (2011)Google Scholar
  4. 4.
    Toreyin, B., Cetin, A.: Online detection of fire in video. In: CVPR (2007)Google Scholar
  5. 5.
    Stauffer, C., Grimson, W.: Learning patterns of activity using real-time tracking. PAMI 22(8), 747–757 (2000)CrossRefGoogle Scholar
  6. 6.
    Basharat, A., Gritai, A., Shah, M.: Learning object motion patterns for anomaly detection and improved object detection. In: CVPR (2008)Google Scholar
  7. 7.
    Wang, X., Ma, X., Grimson, E.: Unsupervised activity perception by hierarchical bayesian models. In: CVPR (2007)Google Scholar
  8. 8.
    Kim, J., Grauman, K.: Observe locally, infer globally: A space-time mrf for detecting abnormal activities with incremental updates. In: CVPR (2009)Google Scholar
  9. 9.
    Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: CVPR (2010)Google Scholar
  10. 10.
    Zhao, B., Fei-Fei, L., Xing, E.: Online detection of unusual events in videos via dynamic sparse coding. In: CVPR (2011)Google Scholar
  11. 11.
    Zaharescu, A., Wildes, R.: Anomalous Behaviour Detection Using Spatiotemporal Oriented Energies, Subset Inclusion Histogram Comparison and Event-Driven Processing. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 563–576. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  12. 12.
    Applied Research Network,
  13. 13.
    Jacobs, N., Burgin, W., Speyer, R., Ross, D., Pless, R.: Adventures in archiving and using three years of webcam images. In: CVPR Workshops (2009)Google Scholar
  14. 14.
    Abrams, A., Feder, E., Pless, R.: Exploratory analysis of time-lapse imagery with fast subset PCA. In: WACV (2011)Google Scholar
  15. 15.
    Jacobs, N., Roman, N., Pless, R.: Consistent temporal variations in many outdoor scenes. In: CVPR (2007)Google Scholar
  16. 16.
    Lalonde, J.F., Narasimhan, S.G., Efros, A.A.: What do the sun and the sky tell us about the camera? IJCV 88(1), 24–51 (2010)CrossRefGoogle Scholar
  17. 17.
    Perez, R., Seals, R., Michalsky, J.: All-weather model for sky luminance distribution - preliminary configuration and validation. Solar Energy 50(3), 235–245 (1993)CrossRefGoogle Scholar
  18. 18.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV 20, 91–110 (2004)CrossRefGoogle Scholar
  19. 19.
    Soatto, S.: On the distance between non-stationary time series. In: Modeling, Estimation and Control. Springer (2007)Google Scholar
  20. 20.
    Kristan, M., Leonardis, A., Skočaj, D.: Multivariate online kernel density estimation with gaussian kernels. Pattern Recognition 44(10-11), 2630–2642 (2011)zbMATHCrossRefGoogle Scholar
  21. 21.
    Brox, T., Bregler, C., Malik, J.: Large displacement optical flow. In: CVPR (June 2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Avinash Ravichandran
    • 1
  • Stefano Soatto
    • 1
  1. 1.Vision LabUniversity of California, Los AngelesLos AngelesUSA

Personalised recommendations