Skip to main content
SpringerLink
Log in

FPGA implementation of an efficient similarity-based adaptive window algorithm for real-time stereo matching

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Stereo matching is one of the most widely used algorithms in real-time image processing applications such as positioning systems for mobile robots, three-dimensional building mapping and both recognition, detection and three-dimensional reconstruction of objects. In area-based algorithms, the similarity between one pixel of the left image and one pixel of the right image is measured using a correlation index computed on vicinities of these pixels called correlation windows. To preserve edges, small windows need to be used. On the other hand, for homogeneous areas, large windows are required. Due to only local information is used, matching between primitives is difficult. In this article, FPGA implementing of an efficient similarity-based adaptive window algorithm for dense disparity maps estimation in real-time is described. To evaluate the proposed algorithm's performance, the developed FPGA architecture was simulated via ModelSim-Altera 6.6c using different synthetic stereo pairs and different sizes for correlation window. In addition, the FPGA architecture was implemented in an FPGA Cyclone IIEP2C35F672C6 embedded in an Altera development board DE2. The disparity maps are computed at a rate of 76 frames per second for stereo pairs of 1280 \(\times\) 1024 pixel resolution and a maximum expected disparity equal to 15. The developed FPGA architecture offers better results with respect to most of the real-time area-based stereo matching algorithms reported in the literature, allows increasing the processing speed up to 93,061,120 pixels per second and enables it to be implemented in the majority of the medium gamma FPGA devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Aguilar-González, A., Pérez-Patricio, M., Arias-Estrada, M., Camas-Anzueto, J.L., Hernández-deLeón, H.R., Sánchez-Alegría, A.: An FPGA correlation-edge distance approach for disparity map. In: Proceedings of IEEE International Conference on Electronics, Communications and Computers (CONIELECOMP15), Chulula, pp. 21–28 (2015)

  2. Alba, A., Arce-Santana, E., Aguilar-Ponce, R.M., Campos-Delgado, D.U.: Phase-correlation guided area matching for realtime vision and video encoding. J Real-Time Image Proc. 9, 621–633 (2014)

    Article  Google Scholar 

  3. Asadi, E., Bottasso, C.L.: Delayed fusion for real-time vision-aided inertial navigation. J Real-Time Image Proc. (2013). doi:10.1007/s11554-013-0376-8

  4. Bartczak, B., Koeser, K., Woelk, F., Koch, R.: Extraction of 3d freeform surfaces as visual landmarks for real-time tracking. J Real-Time Image Proc. 2, 81–101 (2007)

    Article  Google Scholar 

  5. Darabiha, A., MacLean, W., Rose, J.: Reconfigurable hardware implementation of a phase-correlation stereo algorithm. J. Mach. Vis. Appl. 17, 116–132 (2006)

    Article  Google Scholar 

  6. Diaz, J., Ros, E., Pelayo, F., Ortigosa, E., Mota, S.: FPGA based real-time opticalflow system. IEEE Trans. Circuits Syst. Video Technol. 16, 274–279 (2006)

    Article  Google Scholar 

  7. Feiyang, C., Hong, Z., Ding, Y., Mingui, S.: Stereo matching by using the global edge constraint. Neurocomputing 131(11), 217–226 (2013)

  8. Fusiello, A., Roberto, V., Trucco, E.: Symmetric stereo with multiple windowing. Int. J. Pattern Recognit. Artif. Intel. 8(14), 1053–1066 (2000)

  9. Georgoulas, C., Andreadis, I.: Fpga based disparity map computation with vergence control. Microprocess. Microsyst. 34, 259–273 (2010)

    Article  Google Scholar 

  10. Georgoulas, C., Andreadis, I.: A real-time fuzzy hardware structure for disparity map computation. J Real-Time Image Proc. 6, 257–273 (2011)

    Article  Google Scholar 

  11. Georgoulas, C., Kotoulas, L., Sirakoulis, G.C., Andreadis, I., Gasteratos, A.: Real-time disparity map computation module. Microprocess. Microsyst. 32, 159–170 (2008)

    Article  Google Scholar 

  12. Gong, M., Yang, Y.H.: Near real-time reliable stereo matching using programmable graphics hardware. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 1, 924–931 (2005)

    Google Scholar 

  13. Granados, S., Barranco, F., Mota, S., Díaz, J., Ros, E.: On-chip semidense representation map for dense visual features driven by attention processes. J Real-Time Image Proc. 9, 171–185 (2014)

    Article  Google Scholar 

  14. Guerra-Filho, G.: An optimal timespace algorithm for dense stereo matching. J Real-Time Image Proc. 7, 69–86 (2012)

    Article  Google Scholar 

  15. Hirschmuller, H.: Improvements in real-time correlation-based stereo vision. In: Proceedings of IEEE workshop on Stereoand Multi-Baseline Vision, Kauai, pp. 141–148 (2001)

  16. Jin, M., Maruyama, T.: A fast and high quality stereo matching algorithm on fpga. Int. Conf. Field Program. Log. Appl. (FPL) 22(31), 507–5010 (2012)

  17. Jin, S., Cho, J., Pham, X.D., Lee, K.M., Park, S.K., Kim, M., Jeon, J.W.: Fpga design and implementation of a real-time stereo vision system. IEEE Trans. Circuits Syst. Video Technol. 20, 15–26 (2010)

    Article  Google Scholar 

  18. Isakova, N., Basak, S., Sonmez, A.C.: FPGA design and implementation of a real-time stereo vision system. Int. Symp. Innov. Intell. Syst. Appl. (INISTA) 125, 15–26 (2012)

    Google Scholar 

  19. Jung, H.Y., Park, H., Park, I.K., Lee, K.M., Lee, S.U.: Stereo reconstruction using high-order likelihoods. Comput. Vis. Image Underst. 125(21), 223–236 (2014)

    Article  Google Scholar 

  20. Kanade, T., Okutomi, M.: A stereo matching algorithm with an adaptive window: theory and experiment. In: Proceedings of the 1991 IEEE International Conference on Robotics and Automation, Sacramento, vol. 16, pp. 920–932 (1991)

  21. Kanade, T., Yoshida, A., Oda, K., Kano, H., Tanaka, M.: A stereo machine for videorate dense depth mapping and its new applications. In: IEEE Computer Society Conference on Vision and Pattern Recognition, San Francisco, vol. 15, pp. 196–202 (1996)

  22. Lee, S., Yi, J., Kim, J.: Real-time stereo vision on a reconfigurable system. Lect. Notes Comput. Sci. Embed. Comput. Syst. 3553, 299–307 (2005)

    Google Scholar 

  23. Lotti, J., Giraudon, G.: Correlation algorithm with adaptive window for aerial image in stereo vision. In: Proceedings of the Image and Signal Processing for Remote Sensing, EUROPTO’94, Rome, vol. 1, pp. 701–703 (1994)

  24. Madeo, S., Pelliccia, R., Salvadori, C., del Rincon, J.M., Nebel, J.C.: An optimized stereo vision implementation for embedded systems: application to rgb and infra-red images. J Real-Time Image Proc. (2014). doi:10.1007/s11554-014-0461-7

  25. Martin, H., Christian, Z., Vincze, M.: A fast stereo matching algorithm suitable for embedded real-time systems. Comput. Vis. Image Underst. 114, 1180–1202 (2010)

  26. Masrani, D., MacLean, W.: A real-time large disparity range stereo-system using fpgas. In: IEEE International Conference on Computer Vision Systems, pp. 13–19 (2006)

  27. Min, D., Lu, J., Do, M.N.: Joint histogram-based cost aggregation for stereo matching. IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2539–2545 (2013)

    Article  Google Scholar 

  28. Murphy, C., Lindquist, D., Cecil, A.M.R.T., Leavitt, S., Chang, M.L.: Low-cost stereo vision on an FPGA. Int. Symp. Field-Program. Cust. Comput. Mach. 15(21), 333–334 (2007)

    Google Scholar 

  29. Niitsuma, H., Maruyama, T.: High-speed computation of the optical flow. Lect. Notes Comput. Sci. Image Anal. Process. 3617, 287–295 (2005)

    Article  Google Scholar 

  30. Roh, C., Ha, T., Kim, S., Kim, J.: Symmetrical dense disparity estimation: algorithms and fpgas implementation. In: IEEE International Symposium on Consumer Electronics, pp. 452–456 (2004)

  31. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1), 7–42 (2002)

    Article  MATH  Google Scholar 

  32. Stefania Perri, P.C., Cocorullo, G.: Adaptive census transform: a novel hardware-oriented stereovision algorithm. Comput. Vis. Image Underst. 117, 29–41 (2013)

    Article  Google Scholar 

  33. Stefano, L.D., Marchionni, M., Mattoccia, S.: A fast area-based stereo matching algorithm. Image Vis. Comput. 22(22), 983–1005 (2004)

    Article  Google Scholar 

  34. Ttofis, C., Hadjitheophanous, S., Georghiades, A.S., Theocharides, T.: Edge-directed hardware architecture for real-time disparity map computation. IEEE Trans. Comput. 62, 690–704 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  35. Veksler, O.: Stereo matching by compact windows via minimum ratio cycle. IEEE Trans. Pattern Anal. Mach. Intell. 24(12), 1654 (2002)

    Article  Google Scholar 

  36. Woodfill, J., Herzen, B.V.: Real time stereo vision on the parts reconfigurable computer. IEEE Symp. Field-Program. Cust. Comput. Mach. 5, 201–210 (1997a)

    Google Scholar 

  37. Yang, Q.: A non-local cost aggregation method for stereo matching. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’12), Providence, vol. 1, pp. 1–8 (2012)

  38. Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Proceedings of the Third European Conference on Computer Vision (ECCV’94), Stockholm, pp. 151–158 (1994)

  39. Zhang, Y., Kambhamettu, C.: Stereo matching with segmentation-based cooperation. In: Proceedings of the Seventh European Conference on Computer Vision (ECCV’94), Copenhagen, pp. 556–571 (2002)

Download references

Author information

Authors and Affiliations

  1. Division of Graduate Studies and Research, Instituto Tecnológico de Tuxtla Gutiérrez (ITTG), Carretera Panamericana Km. 1080, Tuxtla Gutiérrez, 29050, Chiapas, Mexico

    Madaín Pérez-Patricio & Abiel Aguilar-González

Authors
  1. Madaín Pérez-Patricio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abiel Aguilar-González
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abiel Aguilar-González.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pérez-Patricio, M., Aguilar-González, A. FPGA implementation of an efficient similarity-based adaptive window algorithm for real-time stereo matching. J Real-Time Image Proc 16, 271–287 (2019). https://doi.org/10.1007/s11554-015-0530-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-015-0530-6

Keywords

Search

Navigation