Skip to main content
SpringerLink
Log in

Real-Time Binocular Vision Implementation on an SoC TMS320C6678 DSP

  • Conference paper
  • First Online:
Computer Vision Systems (ICVS 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11754))

Included in the following conference series:

Abstract

In recent years, computer binocular vision has been commonly utilized to provide depth information for autonomous vehicles. This paper presents an efficient binocular vision system implemented on an SoC TMS320C6678 DSP for real-time depth information extrapolation, where the search range propagates from the bottom of an image to its top. To further improve the stereo matching efficiency, the cost function is factorized into five independent parts. The value of each part is pre-calculated and stored in the DSP memory for direct data indexing. The experimental results illustrate that the proposed algorithm performs in real time, when processing the KITTI stereo datasets with eight cores in parallel.

This work is supported by grants from the Shenzhen Science, Technology and Innovation Commission (JCYJ20170818153518789), National Natural Science Foundation of China (No. 61603376) and Guangdong Innovation and Technology Fund (No. 2018B050502009) awarded to Dr. Lujia Wang. This work is also supported by grants from the Research Grants Council of the Hong Kong SAR Government, China (No. 11210017, No. 16212815, No. 21202816, NSFC U1713211) awarded to Prof. Ming Liu.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Brink, J.A., Arenson, R.L., Grist, T.M., Lewin, J.S., Enzmann, D.: Bits and bytes: the future of radiology lies in informatics and information technology. Eur. Radiol. 27(9), 3647–3651 (2017)

    Article  Google Scholar 

  2. Fan, R., Ai, X., Dahnoun, N.: Road surface 3D reconstruction based on dense subpixel disparity map estimation. IEEE Trans. Image Process. 27(6), 3025–3035 (2018)

    Article  MathSciNet  Google Scholar 

  3. Nordhoff, S.: Mobility 4.0: are consumers ready to adopt google’s self-driving car? Master’s thesis, University of Twente (2014)

    Google Scholar 

  4. Fan, R., Jiao, J., Ye, H., Yu, Y., Pitas, I., Liu, M.: Key ingredients of self-driving cars. arXiv preprint arXiv:1906.02939

  5. Ozgunalp, U., Fan, R., Ai, X., Dahnoun, N.: Multiple lane detection algorithm based on novel dense vanishing point estimation. IEEE Trans. Intell. Transp. Syst. 18(3), 621–632 (2017)

    Article  Google Scholar 

  6. Fan, R., Dahnoun, N.: Real-time stereo vision-based lane detection system. Meas. Sci. Technol. 29(7), 074005 (2018)

    Article  Google Scholar 

  7. Bertozzi, M., Broggi, A.: Gold: a parallel real-time stereo vision system for generic obstacle and lane detection. IEEE Trans. Image Process. 7(1), 62–81 (1998)

    Article  Google Scholar 

  8. Fan, R., Wang, L., Liu, M., Pitas, I.: A robust roll angle estimation algorithm based on gradient descent. arXiv preprint arXiv:1906.01894

  9. Fan, R., Prokhorov, V., Dahnoun, N.: Faster-than-real-time linear lane detection implementation using SoC DSP TMS320C6678. In: Proceedings IEEE International Conference Imaging Systems and Techniques (IST), pp. 306–311, October 2016

    Google Scholar 

  10. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  11. Ihler, A.T., John III, W.F., Willsky, A.S.: Loopy belief propagation: convergence and effects of message errors. J. Mach. Learn. Res. 6, 905–936 (2005)

    MathSciNet  MATH  Google Scholar 

  12. Tappen, M.F., Freeman, W.T.: Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters. In: Proceedings Ninth IEEE International Conference on Computer Vision, p. 900. IEEE (2003)

    Google Scholar 

  13. Mozerov, M.G., van de Weijer, J.: Accurate stereo matching by two-step energy minimization. IEEE Trans. Image Process. 24(3), 1153–1163 (2015)

    Article  MathSciNet  Google Scholar 

  14. Sinha, S.N., Scharstein, D., Szeliski, R.: Efficient high-resolution stereo matching using local plane sweeps. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1582–1589 (2014)

    Google Scholar 

  15. Bleyer, M., Rhemann, C., Rother, C.: Extracting 3D scene-consistent object proposals and depth from stereo images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 467–481. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33715-4_34

    Chapter  Google Scholar 

  16. Šára, R.: Finding the largest unambiguous component of stereo matching. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 900–914. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-47977-5_59

    Chapter  Google Scholar 

  17. Sara, R.: Robust correspondence recognition for computer vision. In: Rizzi, A., Vichi, M. (eds.) COMPSTAT 2006-Proceedings in Computational Statistics, pp. 119–131. Springer, Heidelberg (2006). https://doi.org/10.1007/978-3-7908-1709-6_10

    Chapter  Google Scholar 

  18. Cech, J., Sara, R.: Efficient sampling of disparity space for fast and accurate matching. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8. IEEE (2007)

    Google Scholar 

  19. Spangenberg, R., Langner, T., Rojas, R.: Weighted semi-global matching and center-symmetric census transform for robust driver assistance. In: Wilson, R., Hancock, E., Bors, A., Smith, W. (eds.) CAIP 2013. LNCS, vol. 8048, pp. 34–41. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40246-3_5

    Chapter  Google Scholar 

  20. Miksik, O., Amar, Y., Vineet, V., Pérez, P., Torr, P.H.: Incremental dense multi-modal 3D scene reconstruction. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 908–915. IEEE (2015)

    Google Scholar 

  21. Pillai, S., Ramalingam, S., Leonard, J.J.: High-performance and tunable stereo reconstruction. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 3188–3195. IEEE (2016)

    Google Scholar 

  22. Fan, R., Liu, Y., Bocus, M.J., Wang, L., Liu, M.: Real-time subpixel fast bilateral stereo. In: 2018 IEEE International Conference on Information and Automation (ICIA), pp. 1058–1065. IEEE, August 2018

    Google Scholar 

  23. Fan, R., Jiao, J., Pan, J., Huang, H., Shen, S., Liu, M.: Real-time dense stereo embedded in a UAV for road inspection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)

    Google Scholar 

  24. Zhang, Z.: Advanced stereo vision disparity calculation and obstacle analysis for intelligent vehicles. Ph.D. dissertation, University of Bristol (2013)

    Google Scholar 

  25. Fan, R., Liu, Y., Yang, X., Bocus, M.J., Dahnoun, N., Tancock, S.: Real-time stereo vision for road surface 3-D reconstruction. In: Proceedings of IEEE International Conference Imaging Systems and Techniques (IST), pp. 1–6, October 2018

    Google Scholar 

  26. Ai, X.: Active based range measurement systems and applications. Ph.D. dissertation, University of Bristol (2014)

    Google Scholar 

  27. Fan, R., Dahnoun, N.: Real-time implementation of stereo vision based on optimised normalised cross-correlation and propagated search range on a GPU. In: Proceedings of IEEE International Conference Imaging Systems and Techniques (IST), pp. 1–6, October 2017

    Google Scholar 

  28. Mano, M.M.: Computer System Architecture (2003)

    Google Scholar 

  29. Fan, R.: Real-time computer stereo vision for automotive applications. Ph.D. dissertation, University of Bristol, July 2018

    Google Scholar 

  30. Texas Instruments: Multicore fixed and floating-point digital signal processor. Literature Number SPRS691E (2014)

    Google Scholar 

  31. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354–3361, June 2012

    Google Scholar 

  32. Menze, M., Heipke, C., Geiger, A.: Joint 3D estimation of vehicles and scene flow. In: ISPRS Workshop on Image Sequence Analysis (ISA), vol. 8 (2015)

    Article  Google Scholar 

  33. Menze, M., Geiger, A., Heipke, C.: Object scene flow. ISPRS J. Photogram. Remote Sens. (JPRS) 140, 60–76 (2018)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Hong Kong University of Science and Technology, Kowloon, Hong Kong

    Rui Fan, Jianhao Jiao, Yang Yu & Ming Liu

  2. China Unionpay Data Services Co., Ltd., Shanghai, China

    Sicheng Duanmu

  3. University of Bristol, Bristol, UK

    Yanan Liu & Mohammud Junaid Bocus

  4. Unity Drive, Shenzhen, China

    Yilong Zhu

  5. Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China

    Lujia Wang

Authors
  1. Rui Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sicheng Duanmu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yilong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianhao Jiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mohammud Junaid Bocus
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yang Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Lujia Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Ming Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanan Liu .

Editor information

Editors and Affiliations

  1. Centre for Research and Technology Hellas (CERTH-ITI), Thessaloniki, Greece

    Dimitrios Tzovaras

  2. Centre for Research and Technology Hellas (CERTH-ITI), Thessaloniki, Greece

    Dimitrios Giakoumis

  3. Vienna University of Technology, Vienna, Austria

    Markus Vincze

  4. Foundation for Research and Technology Hellas (FORTH), Heraklion, Greece

    Antonis Argyros

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fan, R. et al. (2019). Real-Time Binocular Vision Implementation on an SoC TMS320C6678 DSP. In: Tzovaras, D., Giakoumis, D., Vincze, M., Argyros, A. (eds) Computer Vision Systems. ICVS 2019. Lecture Notes in Computer Science(), vol 11754. Springer, Cham. https://doi.org/10.1007/978-3-030-34995-0_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-34995-0_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-34994-3

  • Online ISBN: 978-3-030-34995-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation