Abstract
To address the problems of the slow convergence and inefficiency in the existing adaptive PID controllers, we proposed a new adaptive PID controller using the Asynchronous Advantage Actor-Critic (A3C) algorithm. Firstly, the controller can parallel train the multiple agents of the Actor-Critic (AC) structures exploiting the multi-thread asynchronous learning characteristics of the A3C structure. Secondly, in order to achieve the best control effect, each agent uses a multilayer neural network to approach the strategy function and value function to search the best parameter-tuning strategy in continuous action space. The simulation results indicated that our proposed controller can achieve the fast convergence and strong adaptability compared with conventional controllers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adel, T., Abdelkader, C.: A particle swarm optimization approach for optimum design of PID controller for nonlinear systems. In: International Conference on Electrical Engineering and Software Applications, pp. 1–4. IEEE (2013)
Savran, A.: A multivariable predictive fuzzy PID control system. Appl. Soft Comput. 13(5), 2658–2667 (2013)
Jiang, D., Wang, W., Shi, L., Song, H.: A compressive sensing-based approach to end-to-end network traffic reconstruction. IEEE Trans. Netw. Sci. Eng. (2018). https://doi.org/10.1109/tnse.2018.2877597
Jiang, D., Huo, L., Li, Y.: Fine-granularity inference and estimations to network traffic for SDN. PLoS One 13(5), 1–23 (2018)
Zhang, X., Bao, H., Du, J., et al.: Application of a new membership function in nonlinear fuzzy PID controllers with variable gains. Inf. Control 2014(5), 1–7 (2014)
Cao-Cang, L.I., Zhang, C.F.: Adaptive neuron PID control based on minimum resource allocation network. Appl. Res. Comput. 32(1), 167–169 (2015)
Patel, R., Kumar, V.: Multilayer neuro PID controller based on back propagation algorithm. Procedia Comput. Sci. 54, 207–214 (2015)
Wang, X.S., Cheng, Y.H., Wei, S.: A proposal of adaptive PID controller based on reinforcement learning. J. China Univ. Min. Technol. 17(1), 40–44 (2007)
Su, Y., Chen, L., Tang, C., et al.: Evolutionary multi-objective optimization of PID parameters for output voltage regulation in ECPT system based on NSGA-II. Trans. China Electrotech. Soc. 31(19), 106–114 (2016)
Akbarimajd, A.: Reinforcement learning adaptive PID controller for an under-actuated robot arm. Int. J. Integr. Eng. 7(2), 20–27 (2015)
Chen, X.S., Yang, Y.M.: A novel adaptive PID controller based on actor-critic learning. Control Theory Appl. 28(8), 1187–1192 (2011)
Bahdanau, D., Brakel, P., Xu, K., et al.: An actor-critic algorithm for sequence prediction. arXiv preprint arXiv:1607.07086 (2016)
Wang, Z., Bapst, V., Heess, N., et al.: Sample efficient actor-critic with experience replay. arXiv preprint arXiv:1611.01224 (2016)
Mnih, V., Badia, A.P., Mirza, M., et al.: Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1928–1937 (2016)
Jiang, D., Huo, L., Lv, Z., et al.: A joint multi-criteria utility-based network selection approach for vehicle-to-infrastructure networking. IEEE Trans. Intell. Transp. Syst. 19, 3305–3319 (2018)
Liu, Q., et al.: A survey on deep reinforcement learning. Chin. J. Comput. 41(01), 1–27 (2018)
Qin, R., Zeng, S., Li, J.J., et al.: Parallel enterprises resource planning based on deep reinforcement learning. Zidonghua Xuebao/Acta Autom. Sin. 43(9), 1588–1596 (2015)
Liao, F.F., Xiao, J.: Research on self-tuning of PID parameters based on BP neural networks. Acta Simulata Syst. Sin. 07, 1711–1713 (2005)
Guo-Yong, L.I., Chen, X.L.: Neural network self-learning PID controller based on real-coded genetic algorithm. Micromotors Servo Tech. 1, 43–45 (2008)
Sheng, X., Jiang, T., Wang, J., et al.: Speed-feed-forward PID controller design based on BP neural network. J. Comput. Appl. 35(S2), 134–137 (2015)
Ma, L., Cai, Z.X.: Fuzzy adaptive controller based on reinforcement learning. Cent. South Univ. Technol. 29(2), 172–176 (1998)
Liu, Z., Zeng, X., Liu, H., et al.: A heuristic two-layer reinforcement learning algorithm based on BP neural networks. J. Comput. Res. Dev. 52(3), 579–587 (2015)
Xu, X., Zuo, L., Huang, Z.: Reinforcement learning algorithms with function approximation: recent advances and applications. Inf. Sci. 261, 1–31 (2014)
Yang, S.Y., Xu, L.P., Wang, P.J.: Study on PID control of a single inverted pendulum system. Control Eng. China S1, 1711–1713 (2007)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Sun, Q., Ren, H., Duan, Y., Yan, Y. (2019). The Adaptive PID Controlling Algorithm Using Asynchronous Advantage Actor-Critic Learning Method. In: Song, H., Jiang, D. (eds) Simulation Tools and Techniques. SIMUtools 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 295. Springer, Cham. https://doi.org/10.1007/978-3-030-32216-8_48
Download citation
DOI: https://doi.org/10.1007/978-3-030-32216-8_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32215-1
Online ISBN: 978-3-030-32216-8
eBook Packages: Computer ScienceComputer Science (R0)