Real-time stereo matching based on fast belief propagation

Abstract

In this paper, a global optimum stereo matching algorithm based on improved belief propagation is presented which is demonstrated to generate high quality results while maintaining real-time performance. These results are achieved using a foundation based on the hierarchical belief propagation architecture combined with a novel asymmetric occlusion handling model, as well as parallel graphical processing. Compared to the other real-time methods, the experimental results on Middlebury data show the efficiency of our approach.

This is a preview of subscription content, access via your institution.

References

  1. 1

    Scharstein D., Szeliski R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV 47(1–3), 7–42 (2002)

    MATH  Article  Google Scholar 

  2. 2

    Sun J., Zheng N.N., Shum H.Y.: Stereo matching using belief propagation. IEEE Trans. PAMI. 25(7), 787–800 (2003)

    Article  Google Scholar 

  3. 3

    Boykov Y., Veksler O., Zabih R.: Fast approximate energy minimization via graph cuts. IEEE Trans. PAMI. 23(11), 1227–1239 (2001)

    Article  Google Scholar 

  4. 4

    Felzenszwalb P.F., Huttenlocher D.P.: Efficient belief propagation for early vision. IJCV 71(1), 41–54 (2006)

    Article  Google Scholar 

  5. 5

    Yang, Q., Wang, L., Yang, R., Wang, S., Liao, M., Nistér, D.: Real-time global stereo matching using hierarchical belief propagation. In: Proceedings of BMVC (2006)

  6. 6

    Sun, J., Zheng, N.-N., Shum, H.-Y.: Symmetric stereo matching for occlusion handling. In: Proceedings of CVPR, vol. II, pp. 399–406 (2005)

  7. 7

    Yang Q., Wang L., Yang R., Stewénius H., Nistér D.: Stereo Matching with color-weighted correlation, hierarchical belief propagation, and occlusion handling. IEEE Trans. PAMI. 31(3), 492–504 (2009)

    Article  Google Scholar 

  8. 8

    Veksler, O.: Fast variable window for stereo correspondence using integral image. In: Proceedings of CVPR, vol. 1, pp. 556–561 (2003)

  9. 9

    Gong M., Yang R., Wang L., Gong M.: A performance study on different cost aggregation approaches used in real-time stereo matching. IJCV 75(2), 283–296 (2007)

    Article  Google Scholar 

  10. 10

    Criminisi A., Blake A., Rother C.: Efficient dense stereo with occlusions for new view-synthesis by four-state dynamic programming. IJCV 71(1), 89–110 (2007)

    Article  Google Scholar 

  11. 11

    Szeliski R., Zabih R., Scharstein D., Veksler O., Kolmogorov V., Agarwala A., Tappen M.F., Rother C.: A comparative study of energy minimization methods for Markov random fields. IEEE Trans. PAMI. 30(6), 1068–1080 (2008)

    Article  Google Scholar 

  12. 12

    Tappen, M.F., Freeman, W.T.: Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters. In Proceedings of ICCV, vol. II, pp. 900–906 (2003)

  13. 13

    Yu, T., Lin, R.-S., Super, B., Tang, B.: Efficient message representations for belief propagation. In: Proceedings of ICCV (2007)

  14. 14

    Liang, C.-K., Cheng, C.-C., Lai, Y.-C., Chen, L.-G., Chen, H.: Hardware Efficient Belief Propagation. In: Proceedings of CVPR 80–87 (2009)

  15. 15

    Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusion using graph cuts. In Proceedings of ICCV, vol. II, pp. 508–515 (2001)

  16. 16

    Ishikawa, H., Geiger. D.: Occlusions, discontinuities, and epipolar lines in stereo. In: Proceedings of ECCV, pp. 232–248 (1998)

  17. 17

    Bobick A.F., Intille S.S.: Large occlusion stereo. IJCV 33(3), 181–200 (1999)

    Article  Google Scholar 

  18. 18

    Yoon K.-J., Kweon I.-S.: Locally adaptive support-weight approach for visual correspondence search. IEEE Trans. PAMI. 28(4), 650–656 (2006)

    Article  Google Scholar 

  19. 19

    Birchfield S., Tomasi C.: A pixel dissimilarity measure that is insensitive to image Sampling. IEEE Trans. PAMI. 20(4), 401–406 (1998)

    Article  Google Scholar 

  20. 20

    Bishop C.: Pattern Recognition and Machine Learning. Springer, New York (2006)

    Google Scholar 

  21. 21

    Tseng, Y.-C., Chang, N., Chang, T.-S.: Low memory cost block-based belief propagation for stereo correspondence. In Proceedings of ICME, pp. 1415–1418 (2007)

  22. 22

    Kraus, M., Strengert, M.: Pyramid filters based on bilinear interpolation. In: Proceedings of GRAPP 2007, vol. GM/R, pp. 21–28 (2007)

  23. 23

    Min D., Sohn K.: Cost aggregation and occlusion handling with WLS in stereo matching. IEEE Trans. Image Process. 17(8), 1431–1442 (2008)

    MathSciNet  Article  Google Scholar 

  24. 24

    Scharstein, D., Szeliski, R.: Middlebury stereo vision research page (2010). http://vision.midd-lebury.edu-/stereo

  25. 25

    Gupta, R., Cho, S.-Y.: Real-time stereo matching using adaptive binary window. In: Proceedings of 3DPVT (2010)

  26. 26

    Zhang, K., Lu, J., Lafruit, G., Lauwereins, R., Van Gool, L.: Real-time accurate stereo with bitwise fast voting on CUDA. In: Proceedings of ICCVW, pp. 794–800 (2009)

  27. 27

    Tombari, F., Mattoccia, S., Di Stefano, L., Addimanda, E.: Near real-time stereo based on effective cost aggregation. In Proceedings of ICPR, pp. 1–4 (2008)

  28. 28

    Kosov, S., Thormählen, T., Seidel, H.-P.: Accurate real-time disparity estimation with variational methods. In: Proceedings of ISVC, pp. 796–807 (2009)

  29. 29

    Wang, L., Liao, M., Gong, M., Yang, R., Nistér, D.: High-quality real-time stereo using adaptive cost aggregation and dynamic programming. In: Proceedings of 3DPVT, pp. 798–805 (2006)

  30. 30

    Gong M., Yang, Y.-H.: Near real-time reliable stereo matching using programmable graphics hardware. In: Proceedings of CVPR, pp. 924–931 (2005)

  31. 31

    Grauer-Gray, S., Kambhamettu, C.: Hierarchical belief propagation to reduce search space using CUDA for stereo and motion estimation. In Proceedings of WACV, pp. 1–8 (2009)

  32. 32

    Trinh, H., McAllester. D.: Unsupervised learning of stereo vision with monocular cues. In: Proceedings of BMVC (2009)

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Zhigeng Pan.

Electronic Supplementary Material

The Below is the Electronic Supplementary Material.

ESM 1 (WMV 11350 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Xiang, X., Zhang, M., Li, G. et al. Real-time stereo matching based on fast belief propagation. Machine Vision and Applications 23, 1219–1227 (2012). https://doi.org/10.1007/s00138-011-0405-1

Download citation

Keywords

  • Stereo matching
  • Hierarchical belief propagation
  • Occlusion handling