Abstract
The existing robot arm control system has long commissioning time and the control system has poor scope of application. In this paper, the Deep Deterministic Policy Gradient (DDPG) algorithm has been adopted and adapted to control the robot arm to move below the object at any position, thereby enhancing the flexibility of the control algorithm and shortening the adjusting time. In addition, to address the problem that physical production line cannot be utilized directly or provide sufficient data for training deep reinforcement learning agent, this paper constructs a virtual model containing both the robot arm and the object as training environment for the agent. Simulation experiment has been performed with state variables and reward properly designed. As is shown by the results, the control agent trained in this paper show good performance in controlling the robot arm, which in turn confirms the effectiveness of the training algorithm with effective data support of the constructed simulation environment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yin, X.G., Wang, H.P., Wu, G.: Path planning algorithm for bending robots. In: 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 392–395 (2009)
Flacco, F., Luca, A.D., Khatib, O.: Motion control of redundant robots under joint constraints: Saturation in the null space In: 2012 IEEE International Conference on Robotics and Automation, pp. 285–292. IEEE (2012)
Cho, H.C., Song, J.B.: Null space motion control of a redundant robot arm using matrix augmentation and saturation method. In: 12th International Conference on Motion and Vibration Control. Japan Society of Mechanical Engineers (2014)
Li, Y.M., Tong, S.C.: Adaptive fuzzy output-feedback stabilization control for a class of switched nonstrict-feedback nonlinear systems. IEEE Trans. Cybern. 47(4), 1007–1016 (2017)
Yu, X., Lin, Y.: Adaptive backstepping quantized control for a class of nonlinear systems. IEEE Trans. Autom. Control 62(2), 981–985 (2017)
Li, X.J., Yang, G.H.: Adaptive decentralized control for a class of interconnected nonlinear systems via backstepping approach and graph theory. Automatica 76, 87–95 (2017)
Wang, H., Wang, Z., Liu, Y.J., et al.: Fuzzy tracking adaptive control of discrete-time switched nonlinear systems. Fuzzy Sets Syst. 316, 35–48 (2017)
Mnih, V., Kavukcuoglu, K., Silver, D., et al.: Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)
Lillicrap, T.P., Hunt, J.J., Pritzel, A., et al.: Continuous control with deep reinforcement learning. Comput. Sci. 8(6), A187 (2015)
Schulman, J., Levine, S., Abbeel, P., et al.: Trust region policy optimization. Comput. Sci. 37, 1889–1897 (2015)
Mnih, V., Badia, A.P., Mirza, M., et al.: Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1928–1937 (2016)
Schulman, J., Wolski, F., Dhariwal, P., et al.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)
Heess, N., Sriram, S., Lemmon, J., et al.: Emergence of locomotion behaviours in rich environments. arXiv preprint arXiv:1707.02286 (2017)
Zhang, Y., Li, W., Zhang, Z.: Physical-limits-constrained minimum velocity norm coordinating scheme for wheeled mobile redundant manipulators. Robotica 6, 1325–1350 (2015)
He, W., David, A.O., Yin, Z., et al.: Neural network control of a robotic manipulator with input deadzone and output constraint. IEEE Trans. Syst. Man Cybern. Syst. 46(6), 759–770 (2016)
Xie, J., Liu, G.L., Yan, S.Z., et al.: Study on neural network adaptive control method for uncertain space manipulator. Yuhang Xuebao/J. Astronaut. 31(1), 123–129 (2010)
Ngo, T.Q., Wang, Y.N., Mai, T.L., et al.: Robust adaptive neural-fuzzy network tracking control for robot manipulator. Int. J. Comput. Commun. Control 7(2), 341–352 (2014)
Lee, C.H., Wang, W.C.: Robust adaptive position and force controller design of robot manipulator using fuzzy neural networks. Nonlinear Dyn. 85(1), 343–354 (2016)
Acknowledgment
This work is supported by the National Key R&D Program of China 2008YFB1004005. We are grateful to their financial support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, H. et al. (2019). Robot Arm Control Method of Moving Below Object Based on Deep Reinforcement Learning. In: Tan, G., Lehmann, A., Teo, Y., Cai, W. (eds) Methods and Applications for Modeling and Simulation of Complex Systems. AsiaSim 2019. Communications in Computer and Information Science, vol 1094. Springer, Singapore. https://doi.org/10.1007/978-981-15-1078-6_11
Download citation
DOI: https://doi.org/10.1007/978-981-15-1078-6_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-1077-9
Online ISBN: 978-981-15-1078-6
eBook Packages: Computer ScienceComputer Science (R0)