Extracting 3D Scene-Consistent Object Proposals and Depth from Stereo Images

  • Michael Bleyer
  • Christoph Rhemann
  • Carsten Rother
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7576)


This work combines two active areas of research in computer vision: unsupervised object extraction from a single image, and depth estimation from a stereo image pair. A recent, successful trend in unsupervised object extraction is to exploit so-called “3D scene-consistency”, that is enforcing that objects obey underlying physical constraints of the 3D scene, such as occupancy of 3D space and gravity of objects. Our main contribution is to introduce the concept of 3D scene-consistency into stereo matching. We show that this concept is beneficial for both tasks, object extraction and depth estimation. In particular, we demonstrate that our approach is able to create a large set of 3D scene-consistent object proposals, by varying e.g. the prior on the number of objects. After automatically ranking the proposals we show experimentally that our results are considerably closer to ground truth than state-of-the-art techniques which either use stereo or monocular images. We envision that our method will build the front-end of a future object recognition system for stereo images.


Accuracy Score Stereo Image Stereo Match Stereo Pair Object Extraction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bleyer, M., Rother, C., Kohli, P., Scharstein, D., Sinha, S.: Object stereo - joint stereo matching and object segmentation. In: CVPR 2011 (2011)Google Scholar
  2. 2.
    Izadi, S., Agarwal, A., Criminisi, A., Winn, J., Blake, A., Fitzgibbon, A.: C-slate: Exploring rem. collaboration on horiz. multi-touch surfaces. In: Tabletop (2007)Google Scholar
  3. 3.
    Ladicky, L., Sturgess, P., Russell, C., Sengupta, S., Bastanlar, Y., Clocksin, W., Torr, P.: Joint optimisation for object class segmentation and dense stereo reconstruction. In: BMVC (2010)Google Scholar
  4. 4.
    Price, B., Cohen, S.: Stereocut: Consistent interactive object selection in stereo image pairs. In: ICCV (2011)Google Scholar
  5. 5.
    Björkman, M., Kragic, D.: Active 3d scene segmentation and detection of unknown objects. In: Conference on Robotics and Automation (2010)Google Scholar
  6. 6.
    Björkman, M., Kragic, D.: Active 3d segmentation through fixation of previously unseen objects. In: BMVC (2010)Google Scholar
  7. 7.
    Gupta, A., Efros, A., Hebert, M.: Blocks World Revisited: Image Understanding Using Qualitative Geometry and Mechanics. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 482–496. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  8. 8.
    Ion, A., Carreira, J., Sminchisescu, C.: Image segmentation by discounted cumulative ranking on maximal cliques. In: ICCV (2011)Google Scholar
  9. 9.
    Hoiem, D., Efros, A., Hebert, M.: Putting objects in perspective. In: IJCV (2008)Google Scholar
  10. 10.
    Carreira, J., Sminchisescu, C.: Constrained parametric min-cuts for automatic object segmentation. In: PAMI (2012)Google Scholar
  11. 11.
    Endres, I., Hoiem, D.: Category Independent Object Proposals. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 575–588. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  12. 12.
    Carreira, J., Li, F., Sminchisescu, C.: Object recognition by sequential figure-ground ranking. IJCV (2012)Google Scholar
  13. 13.
    Bleyer, M., Rhemann, C., Rother, C.: Patchmatch stereo - stereo matching with slanted support windows. In: BMVC (2011)Google Scholar
  14. 14.
    Yoon, K., Kweon, I.: Locally adaptive support-weight approach for visual correspondence search. In: CVPR (2005)Google Scholar
  15. 15.
    Rhemann, C., Hosni, A., Bleyer, M., Rother, C., Gelautz, M.: Fast cost-volume filtering for visual correspondence and beyond. In: CVPR (2011)Google Scholar
  16. 16.
    Delong, A., Osokin, A., Isack, H., Boykov, Y.: Fast approximate energy minimization with label costs. In: CVPR (2010)Google Scholar
  17. 17.
    Christoudias, C., Georgescu, B., Meer, P.: Synergism in low-level vision. In: ICPR, vol. 4, pp. 150–155 (2002)Google Scholar
  18. 18.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: From contours to regions: An empirical evaluation. In: CVPR (2009)Google Scholar
  19. 19.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV 47, 7–42 (2002)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Michael Bleyer
    • 1
  • Christoph Rhemann
    • 1
    • 2
  • Carsten Rother
    • 2
  1. 1.Vienna University of TechnologyViennaAustria
  2. 2.Microsoft Research CambridgeCambridgeUK

Personalised recommendations