Skip to main content
SpringerLink
Log in

Neural Networks Based Single Robot Arm Control for Visual Servoing

  • Chapter
  • First Online:
Neural Networks for Cooperative Control of Multiple Robot Arms

Part of the book series: SpringerBriefs in Applied Sciences and Technology ((BRIEFSINTELL))

  • 973 Accesses

Abstract

In this chapter, we investigate the kinematic control of a single robot arm with an eye-in-hand camera for visual servoing by using neural networks. The visual servoing problem is formulated as a constrained quadratic program, which is then solved via a recurrent neural network. By this approach, the visual servoing with respect to a static point object is achieved with the feature coordinate errors in the image space converging to zero. Besides, joint angle and velocity limits of the robot arm are satisfied, which thus enhances the safety of the robot arm during the visual servoing process. The performance of the approach is guaranteed via theoretical analysis and validated via a simulative example.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Li K, Zhang Y (2012) State adjustment of redundant robot manipulator based on quadratic programming. Robotica 30:477–489

    Article  Google Scholar 

  2. Chen Y-J, Ju M-Y, Hwang K-S (2017) A virtual torque-based approach to kinematic control of redundant manipulators. IEEE Trans Ind Electron 64(2)

    Google Scholar 

  3. Xiang J, Zhong C (2012) A varied weights method for the kinematic control of redundant manipulators with multiple constraints. IEEE Trans Robot 28(2):330–340

    Article  Google Scholar 

  4. Kanoun O, Lamiraux F, Wieber P-B (2011) Kinematic control of redundant manipulators: Generalizing the task-priority framework to inequality task. IEEE Trans Robot 27(4):785–792

    Article  Google Scholar 

  5. Guigue A, Ahmadi M, Langlois R, Hayes MJD (2010) Pareto optimality and multiobjective trajectory planning for a 7-DOF redundant manipulator. IEEE Trans Robot 26(6):1094–1099

    Article  Google Scholar 

  6. Mohammed AM, Li S (2016) Dynamic neural networks for kinematic redundancy resolution of parallel stewart platforms. IEEE Trans Cybern 46(7):1538–1550

    Article  Google Scholar 

  7. Li S, He J, Rafique U, Li Y (2016) Distributed recurrent neural networks for cooperative control of manipulators: A game-theoretic perspective. IEEE Trans Neural Netw Learn Syst 28(2):415–426

    Article  MathSciNet  Google Scholar 

  8. Xiao L, Zhang Y (2015) Dynamic design, numerical solution and effective verification of acceleration-level obstacle-avoidance scheme for robot manipulators. Int J Sys Sci 47(4):1–14

    MathSciNet  Google Scholar 

  9. Guo D, Zhang Y (2012) A new inequality-based obstacle-avoidance MVN scheme and its application to redundant robot manipulators. IEEE Trans Syst Man Cybern Part C Appl Rev 42(6):1326–1340

    Article  Google Scholar 

  10. Guo D, Li K, (2016) Acceleration-level obstacle-avoidance scheme for motion planning of redundant robot manipulators. In: IEEE International Conference on Robotics and Biomimetics (ROBIO), pp 1313–1318

    Google Scholar 

  11. Guo D, Zhang Y (2014) Acceleration-level inequality-based MAN scheme for obstacle avoidance of redundant robot manipulators. IEEE Trans Ind Electron 61(12):6903–6914

    Article  Google Scholar 

  12. Zhang Y, Pan Z, Li K, Guo D (2009) More illustrative investigation on window-shaped obstacle avoidance of robot manipulators using a simplified LVI-based primal-dual neural network. In: International Conference on Mechatronics and Automation (ICMA 2009), pp 4240–4245

    Google Scholar 

  13. Zhang Y, Yan X, Chen D, Guo D, Li W (2016) QP-based refined manipulability-maximizing scheme for coordinated motion planning and control of physically constrained wheeled mobile redundant manipulators. Nonlinear Dyn 85(1):245–261

    Article  MathSciNet  Google Scholar 

  14. Jin L, Li S, La HM, Luo X Manipulability optimization of redundant manipulators using dynamic neural networks. IEEE Trans Ind Electron (in press) https://doi.org/10.1109/TIE.2017.2674624

  15. Li K, Zhang Y (2012) Fault-tolerant motion planning and control of redundant manipulator. Control Eng Pract 20(3):282–292

    Article  Google Scholar 

  16. Ben-Gharbia KM, Maciejewski AA, Roberts RG (2013) Kinematic design of redundant robotic manipulators for spatial positioning that are optimally fault tolerant. IEEE Trans Robot 29(5):1300–1307

    Article  Google Scholar 

  17. Zhang Y, Ge SS, Lee TH (2004) A unified quadratic-programming-based dynamical system approach to joint torque optimization of physically constrained redundant manipulators. IEEE Trans Syst Man Cybern Part B-Cybern 34(5):2126–2132

    Article  Google Scholar 

  18. Zhang Z, Beck A, Magnenat-Thalmann N (2015) Human-like behavior generation based on head-arms model for tracking external targets and body parts. IEEE Trans Cybern 45(8):1390–1400

    Article  Google Scholar 

  19. Li S, Zhang Y, Jin L Kinematic control of redundant manipulators using neural networks. IEEE Trans Neural Netw Learn Syst (in press) https://doi.org/10.1109/TNNLS.2016.2574363

  20. Jin L, Zhang Y (2015) G2-type SRMPC scheme for synchronous manipulation of two redundant robot arms. IEEE Trans Cybern 45(2):153–164

    Article  Google Scholar 

  21. Jin L, Li S Distributed task allocation of multiple robots: a control perspective. IEEE Trans Syst Man Cybern Syst (in press) https://doi.org/10.1109/TSMC.2016.2627579

  22. Jin L, Li S, Xiao L, Lu R, Liao B, Cooperative motion generation in a distributed network of redundant robot manipulators with noises. IEEE Trans Syst Man Cybern Syst (in press) https://doi.org/10.1109/TSMC.2017.2693400

  23. Jin L, Zhang Y, Li S, Zhang Y (2016) Modified ZNN for time-varying quadratic programming with inherent tolerance to noises and its application to kinematic redundancy resolution of robot manipulators. IEEE Trans Ind Electron 63(11):6978–6988

    Article  Google Scholar 

  24. Mirza M A, Li S, Jin L, Simultaneous learning and control of parallel Stewart platforms with unknown parameters. Neurocomputing (in press) https://doi.org/10.1016/j.neucom.2017.05.026

  25. Li S, Shao Z, Guan Y, A dynamic neural network approach for efficient control of manipulators. IEEE Trans Syst Man Cybern: Syst (in press) https://doi.org/10.1109/TSMC.2017.2690460

  26. Zhang Y, Li S Predictive suboptimal consensus of multiagent systems with nonlinear dynamics. IEEE Trans Syst Man Cybern Syst (in press) https://doi.org/10.1109/TSMC.2017.2668440

  27. Zhang Y, Li S Time-scale expansion-based approximated optimal control for underactuated systems using projection neural networks. IEEE Trans Syst Man Cybern Syst (in press) https://doi.org/10.1109/TSMC.2017.2703140

  28. Zhang Y, Li S, Liu X, Adaptive near-optimal control of uncertain systems with application to underactuated surface vessels. IEEE Trans Control Syst Technol (in press) https://doi.org/10.1109/TCST.2017.2705057

  29. Liao B, Zhang Y, Jin L (2016) Taylor \(O(h^3)\) discretization of ZNN models for dynamic equality-constrained quadratic programming with application to manipulators. IEEE Trans Neural Netw Learn Syst 27(2):225–237

    Article  MathSciNet  Google Scholar 

  30. Guo D, Nie Z, Yan L, Novel discrete-time Zhang neural network for time-varying matrix inversion. IEEE Trans Syst Man Cybern Syst (in press) https://doi.org/10.1109/TSMC.2017.2656941

  31. Jin L, Zhang Y, Li S, Zhang Y (2017) Noise-tolerant ZNN models for solving time-varying zero-finding problems: A control-theoretic approach. IEEE Trans Autom Control 62(2):992–997

    Article  MATH  MathSciNet  Google Scholar 

  32. Liu M, Liao B, Ding L, Xiao L (2016) Performance analyses of recurrent neural network models exploited for online time-varying nonlinear optimization. Compu Sci Inf Syst 13(2):691–705

    Article  Google Scholar 

  33. Li S, Guo Y (2012) Neural-network based auv path planning in estuary environments. In: The 10th World Congress on Intelligent Control and Automation (WCICA2012). Beijing, China, pp 3724–3730

    Google Scholar 

  34. Li S, Meng M Q H, Chen W, Li Y, Zhou Y (2007) SP-NN: A novel neural network approach for path planning. In: International Conference on Robotics and Biomimetics (ROBIO), Sanya, China, pp 1355–1360

    Google Scholar 

  35. Xiao L (2015) A finite-time convergent neural dynamics for online solution of time-varying linear complex matrix equation. Neurocomputing 167:254–259

    Article  Google Scholar 

  36. Zhang Y, Li S, Guo H A type of biased consensus-based distributed neural network for path planning. Nonlinear Dyn (in press) https://doi.org/10.1007/s11071-017-3553-7

  37. Li S, Chen S, Lou Y, Lu B, Liang Y (2012) A recurrent neural network for inter-localization of mobile phones. In: International Joint Conference on Neural Networks (IJCNN)

    Google Scholar 

  38. Xiao L (2015) A nonlinearly-activated neurodynamic model and its finite-time solution to equality-constrained quadratic optimization with nonstationary coefficients. Appl Soft Comput 40:252–259

    Article  Google Scholar 

  39. Liao B, Zhang Y (2014) Different complex ZFs leading to different complex ZNN models for time-varying complex generalized inverse matrices. IEEE Trans Neural Netw Learn Syst 25(9):1621–1631

    Article  Google Scholar 

  40. Guo D, Nie Z, Yan L (2016) Theoretical analysis, numerical verification and geometrical representation of new three-step DTZD algorithm for time-varying nonlinear equations solving. Neurocomputing 214:516–526

    Article  Google Scholar 

  41. Li S, Zhou M, Luo X, You Z (2016) Distributed winner-take-all in dynamic networks. IEEE Trans Autom Control 62(2):577–589

    Article  MATH  MathSciNet  Google Scholar 

  42. Jin L, Li S Nonconvex function activated zeroing neural network models for dynamic quadratic programming subject to equality and inequality constraints. Neurocomputing. (in press) https://doi.org/10.1016/j.neucom.2017.05.017

  43. Li S, Li Y, Wang Z (2013) A class of finite-time dual neural networks for solving quadratic programming problems and its k-winners-take-all application. Neural Netw 39:27–39

    Article  MATH  Google Scholar 

  44. Li S, Liu B, Chen B, Lou Y (2013) Neural network based mobile phone localization using bluetooth connectivity. Neural Comput Appl 23(3):667–675

    Article  Google Scholar 

  45. Xiao L, Liao B (2016) A convergence-accelerated Zhang neural network and its solution application to Lyapunov equation. Neurocomputing 193:213–218

    Article  Google Scholar 

  46. Li S, Chen S, Liu B (2013) Accelerating a recurrent neural network to finite-time convergence for solving time-varying Sylvester equation by using a sign-bi-power activation function. Neural Process Lett 37:189–205

    Article  Google Scholar 

  47. Li S, Qin F (2013) A dynamic neural network approach for solving nonlinear inequalities defined on a graph and its application to distributed, routing-free, range-free localization of WSNs. Neurocomputing 117:72–80

    Article  Google Scholar 

  48. Jin L, Zhang Y, Li S (2016) Integration-enhanced Zhang neural network for real-time varying matrix inversion in the presence of various kinds of noises. IEEE Trans Neural Netw Learn Syst 27(12):2615–2627

    Article  Google Scholar 

  49. Mao M, Li J, Jin L, Li S, Zhang Y (2016) Enhanced discrete-time Zhang neural network for time-variant matrix inversion in the presence of bias noises. Neurocomputing 207:220–230

    Article  Google Scholar 

  50. Xiao L (2015) A nonlinearly activated neural dynamics and its finite-time solution to time-varying nonlinear equation. Neurocomputing 173:1983–1988

    Article  Google Scholar 

  51. Liu Y, Gao Y, Tong S, Chen CLP (2016) A unified approach to adaptive neural control for nonlinear discrete-time systems with nonlinear dead-zone input. IEEE Trans Neural Netw Learn Syst 27(1):139–150

    Article  MathSciNet  Google Scholar 

  52. Li M, Li Y, Ge SS, Lee TH (2017) Adaptive control of robotic manipulators with unified motion constraints. IEEE Trans Syst Man Cybern Syst 47(1):184–194

    Article  Google Scholar 

  53. Wang H, Chen B, Liu K, Liu X, Lin C (2014) Adaptive neural tracking control for a class of nonstrict-feedback stochastic nonlinear systems with unknown backlash-like hysteresis. IEEE Trans Neural Netw Learn Syst 25(5):947–958

    Article  Google Scholar 

  54. Wang H, Yang H, Liu X, Liu L, Li S (2016) Direct adaptive neural control of nonlinear strict-feedback systems with un-modeled dynamics using small-gain approach. Int J Adapt Control Signal Process 30(6):906–927

    Article  Google Scholar 

  55. Van M, Wu D, Ge SS, Ren H (2016) Fault diagnosis in image-based visual servoing with eye-in-hand configurations using kalman filter. IEEE Trans Ind Electron 12(6):1998–2007

    Google Scholar 

  56. Hutchinson S, Hager GD, Corke PI (1996) Visual servoing with hand-eye manipulator-optimal control approach. IEEE Trans Robot Automation 12(5):651–670

    Article  Google Scholar 

  57. Marchand E, Chaumette F (2002) Virtual visual servoing: a framework for real-time augmented reality. Comput Graph Forum 21(3):289–297

    Article  Google Scholar 

  58. Spong MW, Hutchinson S, Vidyasagar M (2006) Robot Modeling and Control. Wiley, New York, USA

    Google Scholar 

  59. Boyd S, Vandenberghe L (2004) Convex Optimization. Cambridge University Press, England

    Book  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China

    Shuai Li & Yinyan Zhang

Authors
  1. Shuai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yinyan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuai Li .

Rights and permissions

Reprints and permissions

Copyright information

© 2018 The Author(s)

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Li, S., Zhang, Y. (2018). Neural Networks Based Single Robot Arm Control for Visual Servoing. In: Neural Networks for Cooperative Control of Multiple Robot Arms. SpringerBriefs in Applied Sciences and Technology(). Springer, Singapore. https://doi.org/10.1007/978-981-10-7037-2_1

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-7037-2_1

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-7036-5

  • Online ISBN: 978-981-10-7037-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation