Real-Time Stereo Matching Using Memory-Efficient Belief Propagation for High-Definition 3D Tele-Presence Systems

  • Jesús M. Pérez
  • Pablo Sánchez
  • Marcos Martinez
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5856)


High-definition 3D video is one of the features that the next generation of telecommunication systems is exploring. Real-time requirements limit the execution time of stereo-vision techniques to 40-60 milliseconds. Classical belief propagation algorithms (BP) generate high quality depth maps. However, the huge number of required memory accesses limits their applicability in real systems.

This paper proposes a real-time (latency inferior to 40 millisenconds) high-definition (1280x720) stereo matching algorithm using belief propagation. There are two main contributions. The first one is an improved BP algorithm with occlusion, potential errors and texture-less handling that outperforms classical multi-grid bipartite-graph BP while reducing the number of memory accesses. The second one is an adaptive message compression technique with low performance penalty that greatly reduces the memory traffic. The combination of these techniques outperforms classical BP by about 6.0% while reducing the memory traffic by more than 90%.


Stereo-vision Belief propagation High-Definition Real-Time FPGA 


  1. 1.
    Vision project (2009),
  2. 2.
    Diaz, J., Ros, E., Carrillo, R., Prieto, A.: Real-Time System for High-Image Resolution Disparity Estimation. In: IEEE TIP 2007 (2007)Google Scholar
  3. 3.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. In: IEEE CVPR 2004 (2004)Google Scholar
  4. 4.
    Hirschmuller, H.: Stereo Processing by Semiglobal Matching and Mutual Information. In: IEEE TPAMI 2008 (2008)Google Scholar
  5. 5.
    Yang, Q., Wang, L., Yang, R., Wang, S., Liao, M., Nister, D.: Real-time Global Stereo Matching Using Hierarchical Belief Propagation. In: BMCV 2006 (2006)Google Scholar
  6. 6.
    Wang, L., Liao, M., Gong, M., Yang, R., Nister, D.: High-quality real-time stereo using adaptive cost aggregation and dynamic programming. In: IEEE 3DPVT 2006 (2006)Google Scholar
  7. 7.
    Park, S., Jeong, H.: A fast and parallel belief computation structure for stereo matching. In: IASTED IMSA 2007 (2007)Google Scholar
  8. 8.
    Tseng, Y., Chang, N., Chang, T.: Low Memory Cost Block-Based Belief Propagation for Stereo Correspondence. In: IEEE ICME 2007 (2007)Google Scholar
  9. 9.
    Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: IEEE ICCV 2001 (2001)Google Scholar
  10. 10.
    Egnal, G., Wildes, R.: Detecting binocular half occlusions: empirical comparisons of five approaches. In: IEEE PAMI 2002 (2002)Google Scholar
  11. 11.
    Sun, J., Li, Y., Kang, S., Shum, H.: Symmetric stereo matching for occlusion handling. In: IEEE CVPR 2005 (2005)Google Scholar
  12. 12.
    Yang, Q., Engels, C., Akbarzadeh, A.: Near Real-time Stereo for Weakly-Textured Scenes. In: BMCV 2008 (2008)Google Scholar
  13. 13.
    Gong, M., Yang, R.: Image-gradient-guided real-time stereo on graphics hardware. In: IEEE 3DIM 2005 (2005)Google Scholar
  14. 14.
    Yilei, Z., Minglun, G., Yee-Hong, Y.: Local stereo matching with 3D adaptive cost aggregation for slanted surface modeling and sub-pixel accuracy. In: IEEE ICPR 2008 (2008)Google Scholar
  15. 15.
    Yang, Q., Wang, L., Yang, R., Stewenius, H., Nister, D.: Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation, and Occlusion Handling. In: IEEE TPAML 2009 (2009)Google Scholar
  16. 16.
    Huq, S., Koschan, A., Abidi, B., Abidi, M.: Efficient BP Stereo with Automatic Parameter Estimation. In: IEEE ICIP 2008 (2008)Google Scholar
  17. 17.
    Sarkis, M., Diepold, M., Klaus: Sparse stereo matching using belief propagation. In: IEEE ICIP 2008 (2008)Google Scholar
  18. 18.
    Yu, T., Lin Super, R., Bei Tang, B.: Efficient Message Representations for Belief Propagation. In: IEEE ICCV 2007 (2007)Google Scholar
  19. 19.
  20. 20.
    Liang, C., Cheng, C., Lai, Y., Chen, L., Chen, H.: Hardware-Efficient Belief Propagation. In: IEEE CVPR 2009 (2009)Google Scholar
  21. 21.
    Feldmann, M., Mueller, F., Zilly, R., Tanger, K., Mueller, A., Smolic, P., Kauff, T.: Wiegand HHI Test Material for 3D Video, MPEG 2008/M15413, Archans, Framce (April 2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Jesús M. Pérez
    • 1
  • Pablo Sánchez
    • 1
  • Marcos Martinez
    • 2
  1. 1.University of CantabriaSantanderSpain
  2. 2.DS2Paterna,ValenciaSpain

Personalised recommendations