Multidimensional Systems and Signal Processing

, Volume 30, Issue 1, pp 311–342 | Cite as

Anomaly detection with a moving camera using multiscale video analysis

  • Gustavo H. F. de Carvalho
  • Lucas A. ThomazEmail author
  • Allan F. da Silva
  • Eduardo A. B. da Silva
  • Sergio L. Netto


This paper addresses the problem of abandoned object detection in a cluttered environment using a camera moving along a straight track. The developed system compares captured images to a previously recorded reference video, thus requiring proper temporal alignment and geometric registration between the two signals. A real-time constraint is imposed onto the system to allow an effective surveillance capability in practical situations. In this paper, we propose to deal with the simultaneous detection of objects of different sizes using a multiresolution approach together with normalized cross-correlation and a voting step. In order to develop and properly assess the proposed method we designed a database recorded in a real surveillance scenario, consisting of an industrial plant containing a large number of pipes and rotating machines. Also, we have devised a systematic parameter tuning routine that allows the system to be adapted to different scenarios. We have validated it using the designed database. The obtained results are quite effective, achieving real-time, robust abandoned object detection in an industrial plant scenario.


Video surveillance Moving camera Abandoned object detection Cluttered environment 



This work was developed with the partial support of Statoil Brazil, Petrobras, ANP, and CAPES and CNPq funding agencies.


Funding was provided by Conselho Nacional de Desenvolvimento Científico e Tecnológico (Grant No. 203876/2017-2), Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Grant No. 88881.135449/2016-01), Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro.


  1. Chang, L., Zhao, H., Zhai, S., Ma, Y., & Liu, H. (2013). Robust abandoned object detection and analysis based on online learning. In International conference on robotics and biomimetics, Shenzhen, China.Google Scholar
  2. Chen, Y. M., & Bajie, I. V. (2011). A joint approach to global motion estimation and motion segmentation from a coarsely sampled motion vector field. IEEE Transactions on Circuits and Systems for Video Technology, 21(9), 1316–1328.CrossRefGoogle Scholar
  3. Cheng, L., Gong, M., Schuurmans, D., & Caelli, T. (2011). Real-time discriminative background subtraction. IEEE Transactions on Image Processing, 20(5), 1401–1414.MathSciNetCrossRefzbMATHGoogle Scholar
  4. Choi, W., Pantofaru, C., & Savarese, S. (2013). A general framework for tracking multiple people from a moving camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(7), 1577–1591.CrossRefGoogle Scholar
  5. da Silva, A. F., Thomaz, L. A., Carvalho, G., Nakahata, M. T., Jardim, E., de Oliveira, J. F. L., et al. (2014). An annotated video database for abandoned-object detection in a cluttered environment. In International telecommunications symposium, Sao Paulo, Brazil.Google Scholar
  6. da Silva, A. F., Thomaz, L. A., Netto, S. L., & da Silva, E. A. B. (2017). Online video-based sequence synchronization for moving camera object detection. In IEEE international workshop on multimedia signal processing (pp. 1–6). Luton, UK.Google Scholar
  7. Davis, J. W., Morison, A. M., & Woods, D. D. (2007). An adaptive focus-of-attention model for video surveillance and monitoring. Machine Vision and Applications, 18(1), 41–64.CrossRefGoogle Scholar
  8. DeSouza, G. N., & Kak, A. C. (2002). Vision for mobile robot navigation: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(2), 237–267.CrossRefGoogle Scholar
  9. Dore, A., Soto, M., & Regazzoni, C. S. (2010). Bayesian tracking for video analytics. IEEE Signal Processing Magazine, 27(5), 46–55.CrossRefGoogle Scholar
  10. Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010). Object detection with discriminatively trained part-based models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9), 1627–1645.CrossRefGoogle Scholar
  11. FLIR. (2016). Flir GF320. Accessed January 14, 2016.
  12. Ghosh, A., NSubudhi, B., & Ghosh, S. (2012). Object detection from videos captured by moving camera by fuzzy edge incorporated Markov random field and local histogram matching. IEEE Transactions on Circuits and Systems for Video Technology, 22(8), 1127–1135.CrossRefGoogle Scholar
  13. Hartley, R., & Zisserman, A. (2003). Multiple view geometry in computer vision (2nd ed.). Cambridge: Cambridge University Press.zbMATHGoogle Scholar
  14. Hu, W. C., Chen, C. H., Chen, T. Y., Huang, D. Y., & Wu, Z. C. (2015). Moving object detection and tracking from video captured by moving camera. Journal of Visual Communication and Image Representation, 30, 164–180.CrossRefGoogle Scholar
  15. iRobot. (2016). iRobot Roomba vacuum cleaning robot. Accessed February 26, 2018.
  16. Jodoin, P. M., Saligrama, V., & Konrad, J. (2012). Behavior subtraction. IEEE Transactions on Image Processing, 21(9), 4244–4255.MathSciNetCrossRefzbMATHGoogle Scholar
  17. Kim, J., Ye, G., & Kim, D. (2010). Moving object detection under free-moving camera. In IEEE international conference on image processing, Hong Kong.Google Scholar
  18. Kim, C., & Hwang, J. N. (2002). Fast and automatic video object segmentation and tracking for content-based applications. IEEE Transactions on Circuits and Systems for Video Technology, 12(2), 122–129.CrossRefGoogle Scholar
  19. Kim, S., Yun, K., Yi, K., Kim, S., & Choi, J. (2013). Detection of moving objects with a moving camera using non-panoramic background model. Machine Vision and Applications, 24(5), 1015–1028.CrossRefGoogle Scholar
  20. Kong, H., Audibert, J. Y., & Ponce, J. (2010). Detecting abandoned objects with a moving camera. IEEE Transactions on Image Processing, 19(8), 2201–2210.MathSciNetCrossRefzbMATHGoogle Scholar
  21. Kucharczak, F., da Silva, A. F., Thomaz, L. A., Carvalho, G., da Silva, E. A. B., & Netto, S. L. (2014). Comparison and optimization of image descriptors for real-time detection of abandoned objects. In Simpósio de Processamento de Sinais da UNICAMP, Campinas, Brazil.
  22. Kundu, A., Jawahar, C. V., & Krishna, K. M. (2010). Realtime moving object detection from a freely moving monocular camera. In IEEE international conference on robotics and biomimetics (pp. 1635–1640). Tianjin, China.Google Scholar
  23. Lee, S., Yun, I. D., & Lee, S. U. (2010). Robust bilayer video segmentation by adaptive propagation of global shape and local appearance. Journal of Visual Communication and Image Representation, 21(7), 665–676.CrossRefGoogle Scholar
  24. Lin, Y., Tong, Y., Cao, Y., Zhou, Y., & Wang, S. (2017). Visual-attention based background modeling for detecting infrequently moving objects. IEEE Transactions on Circuits and Systems for Video Technology, 27(6), 1208–1221.CrossRefGoogle Scholar
  25. Li, H., Tang, J., Wu, S., Zhang, Y., & Lin, S. (2010). Automatic detection and analysis of player action in moving background sports video sequences. IEEE Transactions on Circuits and Systems for Video Technology, 20(3), 351–364.CrossRefGoogle Scholar
  26. Menezes, P., Lerasle, F., & Dias, J. (2011). Towards human motion capture from a camera mounted on a mobile robot. Image and Vision Computing, 29(6), 382–393.CrossRefGoogle Scholar
  27. Micheloni, C., & Foresti, G. L. (2006). Real-time image processing for active monitoring of wide areas. Journal of Visual Communication and Image Representation, 17(3), 589–604.CrossRefGoogle Scholar
  28. Mukojima, H., Deguchi, D., Kawanishi, Y., & Ide, I. (2016). Moving camera background subtraction for obstacle detection on railway tracks. In International conference on image processing, Arizona, USA.Google Scholar
  29. Nakahata, M. T., Thomaz, L. A., da Silva, A. F., da Silva, E. A. B., & Netto, S. L. (2017). Anomaly detection with a moving camera using spatio-temporal codebooks. Multidimensional Systems and Signal Processing.
  30. Nawaz, T., Poiesi, F., & Cavallaro, A. (2014). Measures of effective video tracking. IEEE Transactions on Image Processing, 23(1), 376–388.MathSciNetCrossRefzbMATHGoogle Scholar
  31. Nordlund, P., & Uhlin, T. (1996). Closing the loop: Detection and pursuit of a moving object by a moving observer. Image and Vision Computing, 14(4), 265–275.CrossRefGoogle Scholar
  32. Pinto, A. M., Correia, M. V., Moreira, A. P., & Costa, P. G. (2014). Unsupervised flow-based motion analysis for an autonomous moving system. Image and Vision Computing, 32(6–7), 391–404.CrossRefGoogle Scholar
  33. Romanoni, A., Matteucci, M., & Sorrenti, D. G. (2014). Background subtraction by combining temporal and spatio-temporal histograms in the presence of camera movement. Machine Vision and Applications, 25(6), 1573–1584.CrossRefGoogle Scholar
  34. Saligrama, V., Konrad, J., & Jodoin, P. M. (2010). Video anomaly identification. IEEE Signal Processing Magazine, 27(5), 18–33.CrossRefGoogle Scholar
  35. Soille, P. (2003). Morphological image analysis: Principles and applications (2nd ed.). Berlin: Springer.zbMATHGoogle Scholar
  36. Subudhi, B. N., Nanda, P. K., & Ghosh, A. (2011). A change information based fast algorithm for video object detection and tracking. IEEE Transactions on Circuits and Systems for Video Technology, 21(7), 993–1004.CrossRefGoogle Scholar
  37. Suhr, J. K., Jung, H. G., Li, G., Noh, S. I., & Kim, J. (2011). Background compensation for pan-tilt-zoom cameras using 1-D feature matching and outlier rejection. IEEE Transactions on Circuits and Systems for Video Technology, 21(3), 371–377.CrossRefGoogle Scholar
  38. Sun, S. W., Wang, Y. C. F., Huang, F., & Liao, H. Y. M. (2013). Moving foreground object detection via robust SIFT trajectories. Journal of Visual Communication and Image Representation, 24(3), 232–243.CrossRefGoogle Scholar
  39. Taneja, A., Ballan, L., & Pollefeys, M. (2015). Geometric change detection in urban environments using images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(11), 2193–2206.CrossRefGoogle Scholar
  40. Tian, Y., Feris, R., Liu, H., Humpapur, A., & Sun, M. T. (2011). Robust detection of abandoned and removed objects in complex surveillance videos. IEEE Transactions on Systems, Man, and Cybernetics, 41(5), 565–576.CrossRefGoogle Scholar
  41. Tomioka, Y., Takara, A., & Kitazawa, H. (2012). Generation of an optimum patrol course for mobile surveillance camera. IEEE Transactions on Circuits and Systems for Video Technology, 22(2), 216–224.CrossRefGoogle Scholar
  42. VDAO. (2014). VDAO—Video database of abandoned objects in a cluttered industrial environment. Accessed February 26, 2018.
  43. VDAO-200. (2017). 200-frame excerpts form VDAO database. Accessed February 26, 2018.
  44. Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C. (2013). Deppflow: Large displacement optical flow with deep matching. In International conference on computer vision, Sydney, Australia.Google Scholar
  45. Xie, Y., Lin, L., & Jia, Y. (2010). Tracking objects with adaptive feature patches for PTZ camera visual surveillance. In International conference on pattern recognition (pp. 1739–1742). Istanbul, Turkey.Google Scholar
  46. Xie, C., Tan, J., Chen, P., Zhang, J., & He, L. (2014). Multi-scale patch-based sparse appearance model for robust object tracking. Machine Vision and Applications, 25(7), 1859–1876.CrossRefGoogle Scholar
  47. Xue, K., Ogunmakin, G., Liu, Y., Vela, P. A., & Wang, Y. (2011). PTZ camera-based adaptive panoramic and multi-layered background model. In IEEE international conference on image processing, Brussels, Belgium.Google Scholar
  48. Xue, K., Liu, Y., Ogunmakin, G., Chen, J., & Zhang, J. (2013). Panoramic Gaussian mixture model and large-scale range background substraction method for PTZ camera-based surveillance systems. Machine Vision and Applications, 24(3), 477–492.CrossRefGoogle Scholar
  49. Yilmaz, A. (2011). Kernel-based object tracking using asymmetric kernels with adaptive scale and orientation selection. Machine Vision and Applications, 22(2), 255–268.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Electrical Engineering ProgramFederal University of Rio de JaneiroRio de JaneiroBrazil

Personalised recommendations