Skip to main content
SpringerLink
Log in
Book cover

Computer Vision in Control Systems-4 pp 105–138Cite as

Development of Fast Parallel Algorithms Based on Visual and Audio Information in Motion Control Systems of Mobile Robots

  • Chapter
  • First Online:

  • 869 Accesses

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 136))

Abstract

Decision making for movement is one of the essential activities in motion control systems of mobile robots. It is based on methods and algorithms of data processing obtained from the mobile robot sensors, usually video and audio sensors, like video cameras and microphone arrays. After image processing, information about the objects and persons including their current positions in area of mobile robot observation can be obtained. The aim of methods and algorithms is to achieve the appropriate precision and effectiveness of mobile robot’s visual perception, as well as the detection and tracking of objects and persons applying the mobile robot motion path planning. The precision in special cases of visual speaking person’s detection and tracking can be augmented adding the information of sound arrival in order to receive and execute the voice commands. There exist algorithms using only visual perception and attention or also the joined audio perception and attention. These algorithms are usually tested in the most cases as simulations and cannot provide a real time tracking objects and people. Therefore, the goal in this chapter is to develop and test the fast parallel algorithms for decision making in the motion control systems of mobile robots. The depth analysis of the existing methods and algorithms was conducted, which provided the main ways to increase the speed of an algorithm, such as the optimization, simplification of calculations, applying high level programming languages, special libraries for image and audio signal processing based on the hybrid hardware and software implementations, using processors like Digital Signal Processor (DSP) and Field-Programmable Gate Array (FPGA). The high speed proposed algorithms were implemented in the parallel computing multiprocessor hardware structure and software platform using the well known NVIDIA GPU processor and GUDA platform, respectively. The experimental results with different parallel structures confirm the real time execution of algorithms for the objects and speaking person’s detection and tracking using the given mobile robot construction.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Siegwart, R., Nourbakhsh, I.R.: Introduction to Autonomous Mobile Robots. The MIT Press (2004)

    Google Scholar 

  2. Erdmann, M., Hsu, D., Overmars, M., Van der Stappen, A.F. (eds.): Algorithmic Foundations of Robotics VI. Springer, Heidelberg (2005)

    Google Scholar 

  3. Siciliano, Br., Khatib, O. (eds.): Springer Handbook of Robotics. Springer, Heidelberg (2008)

    Google Scholar 

  4. Dehkharghani, S.S., Bekiarski, A., Pleshkova, S.: Application of probabilistic methods in mobile robots audio visual motion control combined with laser range finder distance measurements. In: 11th WSEAS International Conference on Circuits, Systems, Electronics, Control & Signal Processing (CSECS’2012), pp. 91–98 (2012)

    Google Scholar 

  5. Dehkharghani, S.S., Bekiarski, A., Pleshkova, S.: Method and algorithm for precise estimation of joined audio visual robot control. In: Iran’s 3rd International Conference on Industrial Automation (2013)

    Google Scholar 

  6. Venkov, P., Bekiarski, Al., Dehkharghani, S.S., Pleshkova, Sn.: Search and tracking of targets with mobile robot by using audio-visual information. In: International Conference on Automation and Informatics (CAI’2010), pp. 463–469 (2010)

    Google Scholar 

  7. Al, Bekiarski: Visual mobile robots perception for motion control. In: Kountchev, R., Nakamatsu, K. (eds.) Advances in Reasoning-Based Image Processing Intelligent Systems, ISRL, vol. 29, pp. 173–209. Springer, Berlin (2012)

    Chapter  Google Scholar 

  8. Pleshkova, Sn., Bekiarski, Al.: Audio visual attention models in the mobile robots navigation. In: Kountchev, R., Nakamatsu, K. (eds.) New Approaches in Intelligent Image Analysis, ISRL, vol. 108, pp. 253–294. Springer International Publishing, Switzerland (2016)

    Google Scholar 

  9. Dehkharghani, S.S., Pleshkova, S.: Geometric thermal infrared camera calibration for target tracking by a mobile robot. Comptes rendus de l’Academie bulgare des Sciences 67(1), 109–114 (2014)

    Google Scholar 

  10. Bekiarski, Al., Pleshkova, Sn.: Microphone array beamforming for mobile robot. In: 8th WSEAS International Conference on Circuits, Systems, Electronics, Control & Signal Processing (CSECS’2009), pp. 146–149 (2009)

    Google Scholar 

  11. Dehkharghani, S.S.: Development of methods and algorithms for audio-visual mobile robot motion control. Doctoral Thesis, Technical University, Sofia, Bulgaria (2013)

    Google Scholar 

  12. Cebenoyan, C.: GPU Computing for Games. Available from: https://developer.nvidia.com/sites/default/files/akamai/gameworks/CN/computing_cgdc_en.pdf. Accessed 6 May 2017

  13. NVIDIA CUDA Compute Unified Device Architecture. Programming Guide. Version 1.0. Available from: http://developer.download.nvidia.com/compute/cuda/1.0/NVIDIA_CUDA_Programming_Guide_1.0.pdf. Accessed 6 May 2017

  14. Nukada, A., Matsuoka, S.: Auto-tuning 3-D FFT library for CUDA GPUs. In: Conference for High Performance Computing Networking, Storage and Analysis, pp. 30.1–30.10 (2009)

    Google Scholar 

  15. Kijsipongse, E., U-ruekolan, S., Ngamphiw, C., Tongsima, S.: Efficient large Pearson correlation matrix computing using hybrid MPI/CUDA. In: Computer Science and Software Engineering (JCSSE’2011), pp. 237–241 (2011)

    Google Scholar 

  16. Podlozhnyuk, V.: Image Convolution with CUDA. NVIDIA. Available from: http://igm.univ-mlv.fr/~biri/Enseignement/MII2/Donnees/convolutionSeparable.pdf. Accessed 6 May 2017

  17. HPC wire. Available from: https://www.hpcwire.com/2008/09/10/compilers_and_more_gpu_architecture_applications/. Accessed 6 May 2017

  18. Cypher, R., Sanz, J.L.C.: The SIMD Model of Parallel Computation. Springer Publishing Company, Incorporated (2011)

    Google Scholar 

  19. NVIDIA CUDA Architecture. Introduction & Overview. Version 1.1 Available from: http://developer.download.nvidia.com/compute/cuda/docs/CUDA_Architecture_Overview.pdf. Accessed 6 May 2017

  20. Sinha, R.S., Singh, S., Singh, S., Banga, V.K.: Accelerating genetic algorithm using general purpose GPU and CUDA. International Journal of Computer Graphics 7(1), 17–30 (2016)

    Article  Google Scholar 

  21. NVIDIA Developer CUDA Toolkit Documentation. Available from: http://docs.nvidia.com/cuda/cuda-samples/index.html#matrix-multiplication. Accessed 6 May 2017

  22. Ding, C., Karlsson, C., Liu, H., Davies, T., Chen, Z.: Matrix multiplication on GPUs with on-line fault tolerance. In: 9th IEEE Int Symposium Parallel and Distributed Processing with Applications, IEEE Compute Society, pp. 311–317 (2011)

    Google Scholar 

  23. Restocchi, V.: Matrix-matrix multiplication on the GPU with NVIDIA CUDA. Available from: https://www.quantstart.com/articles/Matrix-Matrix-Multiplication-on-the-GPU-with-Nvidia-CUDA (2015). Accessed 6 May 2017

  24. Fatahalian, K., Sugerman, J., Hanrahan, P.: Understanding the Efficiency of GPU algorithms for matrix-matrix multiplication. In: Akenine-Möller, T., McCool, M. (eds.) Graphics Hardware: The Eurographics Association, Stanford University, pp. 1–5 (2004)

    Google Scholar 

  25. CUDA—Matrix Multiplication. Available from: https://rohitnarurkar.wordpress.com/2013/11/02/cuda-matrix-multiplication/ Accessed 6 May 2017

  26. Ramey, W.: Languages, APIs and Development Tools for GPU Computing. NVIDIA Corporation. Available from: http://www.nvidia.com/content/GTC/documents/SC09_Languages_DevTools_Ramey.pdf. Accessed 6 May 2017

  27. Java bindings for CUDA. Available from: http://jcuda.org/ Accessed 6 May 2017

  28. GPU Programming in MATLAB. https://www.mathworks.com/company/newsletters/articles/gpu-programming-in-matlab.html. Accessed 6 May 2017

  29. NVIDIA Developer CUDA Toolkit Documentation. http://docs.nvidia.com/cuda/cuda-samples/index.html#simple-cufft. Accessed 6 May 2017

  30. Gembris, D., Neeb, M., Gipp, M., Kugel, A., Männer, R.: Correlation analysis on GPU systems using NVIDIA’s CUDA. J. Real-Time Image Proc. 6(4), 275–280 (2011)

    Article  Google Scholar 

  31. Papamakariosa, G., Rizosa, G., Pitsianisab, N.P., Sunb, X.: Fast computation of local correlation coefficients on graphics processing units. In: Luk, F.T., Schmalz, M.S., Ritter, G.X., Barrera, J., Astola, J.T. (eds.) Mathematics for Signal and Information Processing. Proceedings of SPIE, vol. 7444, pp. 12.1–12.8 (2010)

    Google Scholar 

  32. Podlozhnyuk V. Image Convolution with CUDA. Available from: http://igm.univ-mlv.fr/~biri/Enseignement/MII2/Donnees/convolutionSeparable.pdf. Accessed 6 May 2017

  33. Iandola, F., Sheffield, D., Anderson, M., Phothilimthana, P.M., Keutzer, K.: Communication-minimizing 2D convolution in GPU registers. In: IEEE International Conference on Image Processing (ICIP’2013), pp. 2116–2120 (2013)

    Google Scholar 

  34. Hizook (Robotics news for Academics and professionals). Available from: http://www.hizook.com/blog/2010/03/28/low-cost-depth-cameras-aka-ranging-cameras-or-rgb-d-cameras-emerge-2010. Accessed 6 May 2017

  35. Hokuyo. Available from: https://www.hokuyo-aut.jp/02sensor/07scanner/urg_04lx_ug01.html. Accessed 6 May 2017

  36. Pomerleau, D.A. Neural Nertwork perception for mobile robot guidance. Springer Science+Business Media LLC (1993)

    Google Scholar 

  37. Wang, Zh., Dissanayake, G., Huang, Sh.: Simultaneous Localization and Mapping: Exactly Sparse Information Filters. World Scientific Publishing Co. Pte. Ltd (2011)

    Google Scholar 

  38. Liénard, J.S.: Variability, ambiguity and attention: a perception model based on analog induction. In: Cantoni, V., di Gesù, V., Setti, A., Tegolo, D. (eds.) Attention in Human and Machine Perception 2: Emergence, Attention, and Creativity. Springer Science+Business Media LLC, pp. 87–99 (1999)

    Google Scholar 

  39. Levorato, R., Pagello, E.: DOA acoustic source localization in mobile robot sensor networks. IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC’2015), pp. 71–76 (2015)

    Google Scholar 

  40. Chou, Y., Liu, J.: A robotic indoor 3D mapping system using a 2D laser range finder mounted on a rotating four-bar linkage of a mobile platform. Int. J. Adv. Rob. Syst. 10, 257–271 (2013)

    Article  Google Scholar 

  41. Martínez-Gómez, J., Fernández-Caballero, A., García-Varea, I., Rodríguez, L., Romero-González, Cr.: A taxonomy of vision systems for ground mobile robots. Int. J. Adv. Robot. Syst. 11.1–11.11 (2014)

    Google Scholar 

  42. Naminski, M.R.: An analysis of simultaneous localization and mapping (SLAM) algorithms. Macalester College, Honors Projects Mathematics, Statistics, and Computer Science (2013)

    Google Scholar 

  43. Thrun, S., Burgar, W., Fox, D.: Probabilistic Robotics. Massachusetts Institute of Technology (2006)

    Google Scholar 

  44. Michiel, H.: Jacobian. Encyclopedia of Mathematics. Springer, Berlin (2001)

    Google Scholar 

  45. Dmochowski, Benesty, J.: Direction of arrival estimation using the parameterized spatial correlation matrix. IEEE Trans Audio Speech Lang. Process. 15(4), 1327–1339 (2007)

    Article  Google Scholar 

  46. Chen, J., Huang, Y., Benesty, J.: Time delay estimation. Audio signal processing for next-generation multimedia communication systems. In: Huang, Y., Benesty, J. (eds.) Audio Signal Processing for Next-Generation Multimedia Communication Systems, pp. 197–227. Kluwer Academic Publishers (2004)

    Google Scholar 

  47. Dvorkind, T., Gannot, S.: The time difference of arrival estimation of speech source in a noisy and reverberant environment. Sig. Process. 85(1), 177–204 (2005)

    Article  MATH  Google Scholar 

  48. Vonderschen, K., Wagner, H.: Detecting interaural time differences and remodeling their representation. Trends Neurosci. 37(5), 289–300 (2014)

    Article  Google Scholar 

  49. Tashev, I.: Sound Capture and Processing. Wiley (2009)

    Google Scholar 

  50. Dhull, S., Arya, S., Sahu, O.P.: Comparison of time-delay estimation techniques in acoustic environment. Int. J. Comput. Appl. 8(9), 29–31 (2010)

    Google Scholar 

  51. Dmochowski, J., Benesty, J.: A generalized steered response power method for computationally viable source localization. Trans. Audio Speech Lang. Process. 15(8), 2510–2526 (2007)

    Article  Google Scholar 

  52. DiBiase, J.H.: A high-accuracy, low-latency technique for talker localization in reverberant environments using microphone arrays. Ph.D. thesis, Brown University (2000)

    Google Scholar 

  53. Surveyor SRV-1 Blackfin Robot Review. Available from: http://www.robotreviews.com/reviews/surveyor-srv-1-blackfin-robot-review. Accessed 6 May 2017

  54. Surveyor SRV-1 Blackfin Camera Board. Available from: http://makezine.com/2007/10/26/surveyor-srv1-blackfin-ca/. Accessed 6 May 2017

  55. ST. http://www.st.com/content/st_com/en.html. Accessed 6 May 2017

  56. Scanning range finder URG-04LX-UG01. Available from: http://www.hokuyo-aut.jp/02sensor/07scanner/urg_04lx_ug01.html. Accessed 6 May 2017

  57. Geforce. Available from: http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-560. Accessed 6 May 2017

  58. NVIDIA GeForce 920 M. Available from: https://www.notebookcheck.com/NVIDIA-GeForce-920M.138762.0.html..Accessed 6 May 2017

  59. CUDA Toolkit 8.0. Available from: https://developer.nvidia.com/cuda-downloads/ Accessed 6 May 2017

  60. Microsoft Visual Studio Professional Version 2015. Available from: https://www.visualstudio.com/downloads/. Accessed 6 May 2017

  61. Matlab Release 2016b. Available from: https://www.mathworks.com/products/new_products/latest_features.html. Accessed 6 May 2017

  62. CUDA C Programming Guide. Available from: https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html. Accessed 6 May 2017

  63. MATLAB GPU Computing Support for NVIDIA CUDA-Enabled GPUs. Available from: https://www.mathworks.com/discovery/matlab-gpu.html. Accessed 6 May 2017

  64. Parallel Computing Toolbox. Available from: https://www.mathworks.com/products/parallel-computing/. Accessed 6 May 2017

Download references

Author information

Authors and Affiliations

  1. Technical University, St. Kl. Ohridski 8, 1000, Sofia, Bulgaria

    Sn. Pleshkova & Al. Bekiarski

Authors
  1. Sn. Pleshkova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Al. Bekiarski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sn. Pleshkova .

Editor information

Editors and Affiliations

  1. Institute of Informatics and Telecommunications, Reshetnev Siberian State University of Science and Technology, Krasnoyarsk, Russia

    Margarita N. Favorskaya

  2. Faculty of Education, Science, Technology and Mathematics, University of Canberra, Canberra, Aust Capital Terr, Australia

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Cite this chapter

Pleshkova, S., Bekiarski, A. (2018). Development of Fast Parallel Algorithms Based on Visual and Audio Information in Motion Control Systems of Mobile Robots. In: Favorskaya, M., Jain, L. (eds) Computer Vision in Control Systems-4. Intelligent Systems Reference Library, vol 136. Springer, Cham. https://doi.org/10.1007/978-3-319-67994-5_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-67994-5_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-67993-8

  • Online ISBN: 978-3-319-67994-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation