An Ease-of-Use Stereo-Based Particle Filter for Tracking Under Occlusion

  • Ser-Nam Lim
  • Larry Davis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4814)


We describe a tracker that handles occlusion by clustering foreground pixels based on their disparity values. Stereo-matched foreground pixels, mapped from multiple views to a reference view, allow the tracker to recover occluded foreground regions in the reference view. The stereo algorithm utilizes a common plan view into which foreground regions from multiple views are projected and intersected to construct polygons that contain the ground plane locations of objects, followed by constraining the epipolar search to only those pixels with ground plane locations lying within these polygons. Consequently, a stereo-matched foreground pixel is easily mapped between views by first mapping its ground plane location using pre-computed homography, followed by intersecting the vertical axis passing through the mapped location with the epipolar line. Finally, tracking in a reference view allows a particle filter to be “seamlessly” integrated, so that uncertainties can be effectively dealt with. Experimental results illustrate the effectiveness of our algorithm.


Surveillance Detection Tracking Stereo Multi-camera fusion 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arulampalam, S., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filters for on-line non-linear/non-gaussian bayesian tracking (2002)Google Scholar
  2. 2.
    Elgammal, A., Davis, L.S.: Probabilistic framework for segmenting people under occlusion. In: Proc. of IEEE 8th International Conference on Computer Vision, IEEE Computer Society Press, Los Alamitos (2001)Google Scholar
  3. 3.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)zbMATHGoogle Scholar
  4. 4.
    Irani, M., Anandan, P.: Parallax geometry of pairs of points for 3d scene analysis. In: European Conference on Computer Vision, pp. 17–30 (1996)Google Scholar
  5. 5.
    Irani, M., Anandan, P., Cohen, M.: Direct recovery of planar-parallax from multiple frames. In: Workshop on Vision Algorithms, pp. 85–99 (1999)Google Scholar
  6. 6.
    Irani, M., Anandan, P., Weinshall, D.: From reference frames to reference planes: Multi-view parallax geometry and applications. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 828–845. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  7. 7.
    Isard, M., Blake, A.: Condensation - conditional density propagation for visual tracking. International journal of computer vision 29(1), 5–28 (1998)CrossRefGoogle Scholar
  8. 8.
    Kaucic, R., Perera, A.G.A., Brooksby, G., Kaufhold, J., Hoogs, A.: A unified framework for tracking through occlusions and across sensor gaps. In: CVPR (2005)Google Scholar
  9. 9.
    Khan, S.M., Shah, M.: A multiview approach to tracking people in crowded scenes using a planar homography constraint. In: European Conference on Computer Vision, Graz, Austria (2006)Google Scholar
  10. 10.
    Kim, K., Davis, L.S.: Multi-camera tracking and segmentation of occluded people on ground plane using search-guided particle filtering. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, Springer, Heidelberg (2006)Google Scholar
  11. 11.
    Kuhn, H.W.: The hungarian method for the assignment problem. Naval Research Logistic Quarterly 2, 83–97 (1955)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence (1981)Google Scholar
  13. 13.
    Mittal, A., Davis, L.S.: M2tracker: A multi-view approach to segmenting and tracking people in a cluttered scene using region-based stereo. In: 7th European Conference on Computer Vision, Copenhagen, Denmark (June 2002)Google Scholar
  14. 14.
    Niblack, W., Barber, R., Equitz, W., Flickner, M., Glasman, E., Petkovic, D., Yanker, P., Faloutsos, C.: The qbic project: Querying images by content using color, texture, and shape. Storage and Retrieval for Image and Video Databases, SPIE 1908 (1993)Google Scholar
  15. 15.
    Rabaud, V., Belongie, S.: Counting crowded moving objects. In: CVPR (2006)Google Scholar
  16. 16.
    Sawhney, H.S.: 3d geometry from planar parallax. Computer Vision and Pattern Recognition, 929–934 (1994)Google Scholar
  17. 17.
    Tarabanis, K., Allen, P., Tsai, R.: A survey of sensor planning in computer vision. IEEE Transactions on Robotics and Automation 11(1), 86–104 (1995)CrossRefGoogle Scholar
  18. 18.
    Grimson, W.E.L., Stauffer, C.: Adaptive background mixture models for real-time tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE Computer Society Press, Los Alamitos (1999)Google Scholar
  19. 19.
    Yang, D.B., Gonzalez-Banos, Guibas, L.J.: Counting people in crowds with a real-time network of simple image sensors. In: Ninth IEEE International Conference on Computer Vision, IEEE Computer Society Press, Los Alamitos (2003)Google Scholar
  20. 20.
    Yang, T., Pan, Q., Li, J., Li, S.: Real-time multiple objects tracking with occlusion handling in dynamic scenes. In: CVPR (2005)Google Scholar
  21. 21.
    Zhao, T., Aggarwal, M., Kumar, R., Sawhney, H.: Real-time wide area multi-camera stereo tracking. In: CVPR (2005)Google Scholar
  22. 22.
    Zhao, T., Nevatia, R.: Bayesian human segmentation in crowded situations. In: CVPR (2003)Google Scholar
  23. 23.
    Zhou, Y., Tao, H.: A background layer model for object tracking through occlusion. In: ICCV (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Ser-Nam Lim
    • 1
  • Larry Davis
    • 2
  1. 1.Cognex Corp., Natick, MAUSA
  2. 2.CS Dept., University of Maryland, College Park, MarylandUSA

Personalised recommendations