Object-Level Priors for Stixel Generation

  • Marius CordtsEmail author
  • Lukas Schneider
  • Markus Enzweiler
  • Uwe Franke
  • Stefan Roth
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8753)


This paper presents a stereo vision-based scene model for traffic scenarios. Our approach effectively couples bottom-up image segmentation with object-level knowledge in a sound probabilistic fashion. The relevant scene structure, i.e. obstacles and freespace, is encoded using individual Stixels as building blocks that are computed bottom-up from dense disparity images. We present a principled way to additionally integrate top-down prior information about object location and shape that arises from independent system modules, ranging from geometric cues up to highly confident object detections. This results in an efficient exploration of orthogonal image-based cues, such as disparity and gray-level intensity data, combined in a consistent scene representation. The overall segmentation problem is modeled as a Markov Random Field and solved efficiently through Dynamic Programming.

We demonstrate superior segmentation accuracy compared to state-of-the-art superpixel algorithms regarding obstacles and freespace in the scene, evaluated on a large dataset captured in real-world traffic.


Image Segmentation Markov Random Field Segmentation Accuracy Vehicle Detector Ground Truth Label 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274–2282 (2012)CrossRefGoogle Scholar
  2. 2.
    Arbeláez, P., Hariharan, B., Gu, C.: Semantic segmentation using regions and parts. In: IEEE Conference on Computer Vision and Pattern Recognition (2012)Google Scholar
  3. 3.
    Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 898–916 (2011)CrossRefGoogle Scholar
  4. 4.
    Badino, H., Franke, U., Pfeiffer, D.: The stixel world - a compact medium level representation of the 3D-world. In: Denzler, J., Notni, G., Süße, H. (eds.) DAGM 2009. LNCS, vol. 5748, pp. 51–60. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  5. 5.
    Benenson, R., Mathias, M., Timofte, R., Van Gool, L.: Pedestrian detection at 100 frames per second. In: IEEE Conference on Computer Vision and Pattern Recognition (2012)Google Scholar
  6. 6.
    Carreira, J., Caseiro, R., Batista, J., Sminchisescu, C.: Semantic segmentation with second-order pooling. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VII. LNCS, vol. 7578, pp. 430–443. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  7. 7.
    Dann, C., Gehler, P., Roth, S., Nowozin, S.: Pottics – the potts topic model for semantic image segmentation. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds.) DAGM/OAGM 2012. LNCS, vol. 7476, pp. 397–407. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  8. 8.
    Dollár, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: an evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 34, 743–761 (2012)CrossRefGoogle Scholar
  9. 9.
    Duda, R., Hart, P.: Use of the Hough transformation to detect lines and curves in pictures. Commun. ACM 15(1), 11–15 (1972)CrossRefzbMATHGoogle Scholar
  10. 10.
    Enzweiler, M., Gavrila, D.M.: Monocular pedestrian detection: survey and experiments. IEEE Trans. Pattern Anal. Mach. Intell. 31, 2179–2195 (2009)CrossRefGoogle Scholar
  11. 11.
    Enzweiler, M., Gavrila, D.M.: A multi-level mixture-of-experts framework for pedestrian classification. IEEE Trans. Image Process. 20(10), 2967–2979 (2011)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Enzweiler, M., Hummel, M., Pfeiffer, D., Franke, U.: Efficient Stixel-based object recognition. In: IEEE Intelligent Vehicles Symposium (2012)Google Scholar
  13. 13.
    Erbs, F., Schwarz, B., Franke, U.: From Stixels to objects - a conditional random field based approach. In: IEEE Intelligent Vehicles Symposium (2013)Google Scholar
  14. 14.
    Everingham, M., Gool, L.V., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88, 303–338 (2010)CrossRefGoogle Scholar
  15. 15.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59, 167–181 (2004)CrossRefGoogle Scholar
  16. 16.
    Fulkerson, B., Vedaldi, A., Soatto, S.: Class segmentation and object localization with superpixel neighborhoods. In: International Conference on Computer Vision (2009)Google Scholar
  17. 17.
    Gavrila, D.M.: A Bayesian, exemplar-based approach to hierarchical shape matching. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1408–1421 (2007)CrossRefGoogle Scholar
  18. 18.
    Jain, A., Duin, R., Mao, J.: Statistical pattern recognition: a review. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 4–37 (2000)CrossRefGoogle Scholar
  19. 19.
    Ladický, L., Sturgess, P., Russell, C., Sengupta, S., Bastanlar, Y., Clocksin, W., Torr, P.H.S.: Joint optimisation for object class segmentation and dense stereo reconstruction. In: British Machine Vision Conference (2010)Google Scholar
  20. 20.
    Ladický, L’., Sturgess, P., Alahari, K., Russell, C., Torr, P.H.S.: What, Where and How Many? Combining Object Detectors and CRFs. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 424–437. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  21. 21.
    Muffert, M., Schneider, N., Franke, U.: Stix-Fusion: a probabilistic Stixel integration technique. In: Canadian Conference on Computer and Robot Vision (2014)Google Scholar
  22. 22.
    Pfeiffer, D., Franke, U.: Towards a global optimal multi-layer Stixel representation of dense 3D data. In: British Machine Vision Conference (2011)Google Scholar
  23. 23.
    Scharwächter, T., Enzweiler, M., Franke, U., Roth, S.: Efficient multi-cue scene segmentation. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 435–445. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  24. 24.
    Scharwächter, T., Schuler, M., Franke, U.: Visual guard rail detection for advanced highway assistance systems. In: IEEE Intelligent Vehicles Symposium (2014)Google Scholar
  25. 25.
    Shotton, J., Winn, J., Rother, C., Criminisi, A.: TextonBoost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int. J. Comput. Vis. 81(1), 2–23 (2009)CrossRefGoogle Scholar
  26. 26.
    Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  27. 27.
    Sun, Z., Bebis, G., Miller, R.: On-road vehicle detection: a review. IEEE Trans. Pattern Anal. Mach. Intell. 28, 694–711 (2006)CrossRefGoogle Scholar
  28. 28.
    Viola, P., Jones, M.J.: Robust real-time object detection. Int. J. Comput. Vis. 4, 85–107 (2001)Google Scholar
  29. 29.
    Wojek, C., Schiele, B.: A dynamic conditional random field model for joint labeling of object and scene classes. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 733–747. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  30. 30.
    Zhang, J., Kan, C., Schwing, A.G., Urtasun, R.: Estimating the 3D layout of indoor scenes and its clutter from depth sensors. In: International Conference on Computer Vision (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Marius Cordts
    • 1
    • 2
    Email author
  • Lukas Schneider
    • 1
  • Markus Enzweiler
    • 1
  • Uwe Franke
    • 1
  • Stefan Roth
    • 2
  1. 1.Environment Perception, Daimler R&DSindelfingenGermany
  2. 2.Department of Computer ScienceTU DarmstadtDarmstadtGermany

Personalised recommendations