Mobile Surveillance by 3D-Outlier Analysis

  • Peter Holzer
  • Axel Pinz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6468)


We present a novel online method to model independent foreground motion by using solely traditional structure and motion (S+M) algorithms. On the one hand, the visible static scene can be reconstructed and on the other hand, the position and orientation (pose) of the observer (mobile camera) are estimated. Additionally, we use 3D-outlier analysis for foreground motion detection and tracking. First, we cluster the available 3D-information such that, with high probability, each cluster corresponds to a moving object. Next, we establish a purely geometry-based object representation that can be used to reliably estimate each object’s pose. Finally, we extend the purely geometry-based object representation and add local descriptors to solve the loop closing problem for the underlying S+M algorithm. Experimental results on single and multi-object video data demonstrate the viability of this method. Major results include the computation of a stable representation of moving foreground objects, basic recognition possibilities due to descriptors, and motion trajectories that can be used for motion analysis of objects. Our novel multibody structure and motion (MSaM) approach runs online and can be used to control active surveillance systems in terms of dynamic scenes, observer pose, and observer-to-object pose estimation, or to enrich available information in existing appearance- and shape-based object categorization.


Kalman Filter Point Feature Augmented Reality Active Surveillance System Move Foreground Object 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Nistér, D., Narodistky, O., Bergen, J.: Visual odometry. In: CVPR, pp. 652–659 (2004)Google Scholar
  2. 2.
    Davison, A.J., Reid, I., Molton, N., Stasse, O.: Monoslam: Real-time single camera slam. PAMI 29, 1052–1067 (2007)CrossRefGoogle Scholar
  3. 3.
    Schweighofer, G., Segvic, S., Pinz, A.: Online/realtime structure and motion for general camera models. In: IEEE WACV (2008)Google Scholar
  4. 4.
    Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: ISMAR (2007)Google Scholar
  5. 5.
    Williams, B., Klein, G., Reid, I.: Real-time slam relocalisation. In: ICCV (2007)Google Scholar
  6. 6.
    Klein, G., Murray, D.: Improving the agility of keyframe-based SLAM. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 802–815. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Newcombe, R.A., Davison, A.J.: Live dense reconstruction with a single moving camera. In: CVPR (2010)Google Scholar
  8. 8.
    Schindler, K., Suter, D., Wang, H.: A model-selection framework for multibody structure-and-motion of image sequences. IJCV 79, 159–177 (2008)CrossRefGoogle Scholar
  9. 9.
    Costeira, J., Kanade, T.: A multi-body factorization method for motion analysis. In: ICCV, pp. 1071–1076 (1995)Google Scholar
  10. 10.
    Costeira, J.P., Kanade, T.: A multibody factorization method for independently moving objects. IJCV 29, 159–179 (1998)CrossRefGoogle Scholar
  11. 11.
    Yan, J., Pollefeys, M.: A general framework for motion segmentation: Independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 94–106. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Fitzgibbon, A.W., Zisserman, A.: Multibody structure and motion: 3-D reconstruction of independently moving objects. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 891–906. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  13. 13.
    Li, T., Kallem, V., Singaraju, D., Vidal, R.: Projective factorization of multiple rigid-body motions. In: CVPR (2007)Google Scholar
  14. 14.
    Ozden, K., Schindler, K., Gool, L.V.: Multibody structure-from-motion in practice. PAMI 32, 1134–1141 (2010)CrossRefGoogle Scholar
  15. 15.
    Leibe, B., Schindler, K., Cornelis, N., Gool, L.V.: Coupled object detection and tracking from static cameras and moving vehicles. PAMI 30, 1683–1698 (2008)CrossRefGoogle Scholar
  16. 16.
    Ess, A., Leibe, B., Schindler, K., Gool, L.V.: A mobile vision system for robust multi-person tracking. In: CVPR (2008)Google Scholar
  17. 17.
    Fukunaga, K., Hostetler, L.D.: The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans. on Information Theory 21, 32–40 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. PAMI 24, 603–619 (2002)CrossRefGoogle Scholar
  19. 19.
    Shi, J., Tomasi, C.: Good features to track. In: CVPR, pp. 593–600 (1994)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Peter Holzer
    • 1
  • Axel Pinz
    • 1
  1. 1.Institute of Electrical Measurement and Measurement Signal ProcessingGraz University of TechnologyAustria

Personalised recommendations