Abstract
In this paper, we aim to take mobile multi-object tracking to the next level. Current approaches work in a tracking-by-detection framework, which limits them to object categories for which pre-trained detector models are available. In contrast, we propose a novel tracking-before-detection approach that can track both known and unknown object categories in very challenging street scenes. Our approach relies on noisy stereo depth data in order to segment and track objects in 3D. At its core is a novel, compact 3D representation that allows us to robustly track a large variety of objects, while building up models of their 3D shape online. In addition to improving tracking performance, this representation allows us to detect anomalous shapes, such as carried items on a person’s body. We evaluate our approach on several challenging video sequences of busy pedestrian zones and show that it outperforms state-of-the-art approaches.
Chapter PDF
References
Leibe, B., Schindler, K., Van Gool, L.: Coupled Object Detection and Tracking from Static Cameras and Moving Vehicles. PAMI 30(10), 1683–1698 (2008)
Ess, A., Leibe, B., Schindler, K., Van Gool, L.: Robust Multi-Person Tracking from a Mobile Platform. PAMI 31(10), 1831–1846 (2009)
Bajracharya, M., Moghaddam, B., Howard, A., Brennan, S., Matthies, L.: A Fast Stereo-based System for Detecting and Tracking Pedestrians from a Moving Vehicle. IJRS 28(11-12), 1466–1485 (2009)
Bansal, M., Jung, S.H., Matei, B., Eledath, J., Sawhney, H.S.: A real-time pedestrian detection system based on structure and appearance classification. In: ICRA (2010)
Kuo, C.H., Huang, C., Nevatia, R.: Multi-Target Tracking by On-Line Learned Discriminative Appearance Models. In: CVPR (2010)
Andriluka, M., Roth, S., Schiele, B.: Monocular 3D Pose Estimation and Tracking by Detection. In: CVPR (2010)
Wojek, C., Walk, S., Roth, S., Schiele, B.: Monocular 3D Scene Understanding with Explicit Occlusion Reasoning. In: CVPR (2011)
Felzenszwalb, P., Girshick, B., McAllester, D., Ramanan, D.: Object Detection with Discriminatively Trained Part-Based Models. PAMI 32(9), 1627–1645 (2010)
Mitzel, D., Horbert, E., Ess, A., Leibe, B.: Multi-person Tracking with Sparse Detection and Continuous Segmentation. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 397–410. Springer, Heidelberg (2010)
Besl, P.J., Mckay, H.D.: A method for registration of 3-D shapes. PAMI 14(2) (1992)
Luber, M., Spinello, L., Arras, K.: People Tracking in RGB-D Data With Online-Boosted Target Models. In: IROS (2011)
Prengaman, R., Thurber, R., Bath, W.: Retrospective Detection Algorithm for Extraction of Weak Targets in Clutter and Interference Environments. In: IEEE Int. Radar Conf. (1982)
Salmond, D., Birch, H.: A Particle Filter for Track-Before-Detect. In: American Control Conf. (2001)
Danescu, R., Oniga, F., Nedevschi, S.: Modeling and Tracking the Driving Environment With a Particle-Based Occupancy Grid. IEEE Trans. Intel. Transp. Syst. 12(4), 1331–1342 (2011)
Geronimo, D., Lopez, A., Sappa, A., Graf, T.: Survey of Pedestrian Detection for Advanced Driver Assistance Systems. PAMI 32(7), 1239–1258 (2010)
Petrovskaya, A., Thrun, S.: Model Based Vehicle Detection and Tracking for Autonomous Urban Driving. AR 26(2–3), 123–139 (2009)
Kaestner, R., Maye, J., Siegwart, R.: Generative Object Detection and Tracking in 3D Range Data. In: ICRA (2012)
Gavrila, D., Munder, S.: Multi-Cue Pedestrian Detection and Tracking from a Moving Vehicle. IJCV 73(1), 41–59 (2007)
Keller, C., Fernandez-Llorca, D., Gavrila, D.: Dense Stereo-Based ROI Generation for Pedestrian Detection. In: Denzler, J., Notni, G., Süße, H. (eds.) Pattern Recognition. LNCS, vol. 5748, pp. 81–90. Springer, Heidelberg (2009)
Mitzel, D., Leibe, B.: Real-Time Multi-Person Tracking with Detector Assisted Structure Propagation. In: ICCV CORP Workshop (2011)
Feldman, A., Hybinette, M., Balch, T., Cavallaro, R.: The Multi-ICP Tracker: An Online Algorithm for Tracking Multiple Interacting Targets. J. Field Robotics (2012)
Teichman, A., Thrun, S.: Tracking-Based Semi-Supervised Learning. In: RSS (2011)
Teichman, A., Levinson, J., Thrun, S.: Towards 3D Object Recognition via Classification of Arbitrary Object Tracks. In: ICRA (2011)
Bjorkman, M., Kragic, D.: Active 3D Segmentation and Detection of Unknown Objects. In: IROS (2010)
Damen, D., Hogg, D.: Detecting Carried Objects in Short Video Sequences. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 154–167. Springer, Heidelberg (2008)
Nistér, D., Naroditsky, O., Bergen, J.: Visual odometry. In: CVPR (2004)
Vedaldi, A., Soatto, S.: Quick Shift and Kernel Methods for Mode Seeking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 705–718. Springer, Heidelberg (2008)
Kaucic, R., Perera, A., Brooksby, G., Kaufhold, J., Hoogs, A.: A Unified Framework for Tracking through Occlusions and Across Sensor Gaps. In: CVPR (2005)
Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive Foreground Extraction Using Iterated Graph Cuts. In: SIGGRAPH (2004)
Boykov, Y., Veksler, O., Zabih, R.: Fast Approximate Energy Minimization via Graph Cuts. PAMI 23(11), 1222–1239 (2001)
Geiger, A., Roser, M., Urtasun, R.: Efficient Large-Scale Stereo Matching. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part I. LNCS, vol. 6492, pp. 25–38. Springer, Heidelberg (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mitzel, D., Leibe, B. (2012). Taking Mobile Multi-object Tracking to the Next Level: People, Unknown Objects, and Carried Items. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision – ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7576. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33715-4_41
Download citation
DOI: https://doi.org/10.1007/978-3-642-33715-4_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33714-7
Online ISBN: 978-3-642-33715-4
eBook Packages: Computer ScienceComputer Science (R0)