Abstract
We present a strategy that combines color and depth images to detect people in indoor environments. Similarity of image appearance and closeness in 3D position over time yield weights on the edges of a directed graph that we partition greedily into tracklets, sequences of chronologically ordered observations with high edge weights. Each tracklet is assigned the highest score that a Histograms-of-Oriented Gradients (HOG) person detector yields for observations in the tracklet. High-score tracklets are deemed to correspond to people. Our experiments show a significant improvement in both precision and recall when compared to the HOG detector alone.
Thanks to Julian (Mac) Mason for his gentle introduction to the Kinect, including his help for calibrating the sensor and obtaining the first set of images. This work was supported by the Consejo Nacional de Ciencia y Tecnología under Grant No. 25288, the Fulbright Scholarship Board, and the Instituto Politécnico Nacional under Grant No. 20110705 for Joaquín Salas, and the National Science Foundation under Grant No. IIS-1017017 and by the Army Research Office under Grant No. W911NF-10-1-0387 for Carlo Tomasi.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Dalal, N., Triggs, B.: Histograms of Oriented Gradients for Human Detection. In: IEEE Computer Vision and Pattern Recognition, vol. 1, pp. 886–893 (2005)
Dalal, N., Triggs, B., Schmid, C.: Human Detection using Oriented Histograms of Flow and Appearance. In: European Conference on Computer Vision, pp. 428–441 (2006)
Dalal, N.: INRIA Person Database (September 2010), http://pascal.inrialpes.fr/soft/olt/
Elfes, A.: Using Occupancy Grids for Mobile Robot Perception and Navigation. Computer 22(6), 46–57 (2002)
Gavrila, D.: Pedestrian Detection from a Moving Vehicle. In: European Conference on Computer Vision, pp. 37–49 (2000)
Gavrila, D., Giebel, J., Munder, S.: Vision-based Pedestrian Detection: The Protector System. In: Intelligent Vehicles Symposium, pp. 13–18 (2004)
Gavrila, D.: The Visual Analysis of Human Movement: A Survey. Computer Vision and Image Understanding 73(1), 82–98 (1999)
Giles, J.: Inside the Race to Hack the Kinect. The New Scientist 208 (2789) (2010)
Gordon, G., Darrell, T., Harville, M., Woodfill, J.: Background Estimation and Removal based on Range and Color. In: IEEE Computer Vision and Pattern Recognition, p. 2 (1999)
Javed, O., Shafique, K., Rasheed, Z., Shah, M.: Modeling Inter-Camera Space-Time and Appearance Relationships for Tracking Across Non-Overlapping Views. Computer Vision and Image Understanding 109(2), 146–162 (2008)
Johansson, G.: Visual Perception of Biological Motion and a Model for its Analysis. Perceiving Events and Objects 3 (1973)
Kelly, M.: Visual Identification of People by Computer. Ph.D. thesis, Stanford University (1971)
Lowe, D.: Object Recognition from Local Scale-invariant Features. In: IEEE International Conference on Computer Vision, p. 1150 (1999)
Micilotta, A., Ong, E., Bowden, R.: Detection and Tracking of Humans by Probabilistic Body Part Assembly. In: British Machine Vision Conference, vol. 1, pp. 429–438 (2005)
Mikolajczyk, K., Schmid, C., Zisserman, A.: Human Detection based on a Probabilistic Assembly of Robust Part Detectors. In: European Conference on Computer Vision, pp. 69–82 (2004)
Mohan, A., Papageorgiou, C., Poggio, T.: Example-based Object Detection in Images by Components. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(4), 349 (2001)
Muñoz, R., Aguirre, E., García, M.: People Detection and Tracking using Stereo Vision and Color. Image and Vision Computing 25(6), 995–1007 (2007)
Nelder, J., Mead, R.: A Simplex Method for Function Minimization. The Computer Journal 7(4), 308 (1965)
Papageorgiou, C., Poggio, T.: A Trainable System for Object Detection. International Journal of Computer Vision 38(1), 15–33 (2000)
Phillips, P.: Human Identification Technical Challenges. In: IEEE International Conference on Image Processing (2002)
Ramanan, D., Forsyth, D., Zisserman, A.: Strike a Pose: Tracking People by Finding Stylized Poses. In: IEEE Computer Vision and Pattern Recognition, pp. 271–278 (2005)
Roberts, T., McKenna, S., Ricketts, I.: Human Pose Estimation using Learnt Probabilistic Region Similarities and Partial Configurations. In: European Conference on Computer Vision, pp. 291–303 (2004)
Ronfard, R., Schmid, C., Triggs, B.: Learning to Parse Pictures of People. In: European Conference on Computer Vision, pp. 700–714 (2006)
Rubner, Y., Tomasi, C., Guibas, L.: The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision 40(2), 99–121 (2000)
Schwartz, W., Kembhavi, A., Harwood, D., Davis, L.: Human Detection using Partial Least Squares Analysis. In: IEEE International Conference on Computer Vision, pp. 24–31 (2010)
Swets, J., Dawes, R., Monahan, J.: Better Decisions through Science. Scientific American, 83 (2000)
Theodoridis, S., Koutroumbas, K.: Pattern Recognition. Elsevier, Amsterdam (2009)
Viola, P., Jones, M.: Rapid Object Detection using a Boosted Cascade of Simple Features. In: IEEE Computer Vision and Pattern Recognition, vol. 1 (2001)
Viola, P., Jones, M., Snow, D.: Detecting Pedestrians using Patterns of Motion and Appearance. International Journal of Computer Vision 63(2), 153–161 (2005)
Vrubel, A., Bellon, O., Silva, L.: Planar Background Elimination in Range Images: A Practical Approach. In: IEEE International Conference on Image Processing, pp. 3197–3200 (2009)
Willow Garage: OpenCV (September 2010), http://opencv.willowgarage.com
Xu, F., Fujimura, K.: Human Detection using Depth and Gray Images. In: IEEE Advanced Video and Signal Based Surveillance. pp. 115–121. IEEE, New York (2003)
Zhao, L., Davis, L.: Closely coupled object detection and segmentation. In: IEEE International Conference on Computer Vision, pp. 454–461 (2005)
Zhao, L., Thorpe, C.: Stereo and Neural Network-based Pedestrian Detection. IEEE Transactions on Intelligent Transportation Systems 1(3), 148–154 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Salas, J., Tomasi, C. (2011). People Detection Using Color and Depth Images. In: Martínez-Trinidad, J.F., Carrasco-Ochoa, J.A., Ben-Youssef Brants, C., Hancock, E.R. (eds) Pattern Recognition. MCPR 2011. Lecture Notes in Computer Science, vol 6718. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21587-2_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-21587-2_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21586-5
Online ISBN: 978-3-642-21587-2
eBook Packages: Computer ScienceComputer Science (R0)