Abstract
In order to get better control performances in a relatively dangerous environment, a novel dynamic optimal CPS control method for interconnected power systems using on-policy reinforcement learning (RL) algorithm-SARSA RL algorithm is introduced in this paper. This controller realizes online learning and optimization of the acceptance rate of CPS values by a reward function which is constructed by the system CPS values and a closed loop which is constructed by CPS control actions. Comparing with off-policy RL algorithm-Q-learning, SARSA is better in convergence ability and safer in selection of policy. It is shown in the simulation experiment that more effective CPS values can be obtained by the controller using SARSA RL algorithm than that by using Q-learning algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Wang Dexing (2000) Study of CPS standards in east china power grid. Autom Elec Power Syst 24(8):41–44
Feliachi A, Rerkpreedapong D (2005) NERC compliant load frequency control design using fuzzy rules. Elec Power Syst Res 73(1):101–106
Yu Tao, Zhou Bin (2009) Reinforcement learning based CPS self-tuning control methodology for interconnected power systems. Power Syst Prot Control 37(10):33–48
Yu Tao, Zhou Bin, Ka Wing Chan (2009) Q-learning based dynamic optimal CPS control for interconnected power grids.Proc CSEE 29(19):13–19
Yu Tao, Yuan Ye (2010) Average reward model based whole process R(λ)-learning for optimal CPS control. Autom Elec Power Syst 34(21):27–33
Watkins CJCH (1989) Learning from delayed rewards. Ph. D. Dissertation, Cambridge University, Cambridge
Tousi MR, Hosseinian SH, Jadidinejad AH, Menhaj MB (2008) Application of SARSA learning algorithm for reactive power control in power system. In: 2nd IEEE international conference on power and energy (PECon 08)
Zhang Rubo Theory and application of reinforcement learning. Harbin Engineering University Press, Haerbin
Gao Yang, Chen Shi-Fu, Lu Xin (2001) Research on reinforcement learning technology: a review. Acta automatica Sinica 30(1):76–90
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge
Imthias Ahamed TP, Nagendra Rao PS, Sastry PS (2002) A reinforcement learning approach to automatic generation control. Elec Power Syst Res 63(1):9–26
Guan Yu, Yang Qi-Yu (2005) On replicated experimental units of design of experiments. J Biomath 20(3):369–374
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media New York
About this paper
Cite this paper
Yu, T., Zhang, S., Hong, Y. (2014). Dynamic Optimal CPS Control for Interconnected Power Systems Based on SARSA Algorithm. In: Xing, S., Chen, S., Wei, Z., Xia, J. (eds) Unifying Electrical Engineering and Electronics Engineering. Lecture Notes in Electrical Engineering, vol 238. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4981-2_29
Download citation
DOI: https://doi.org/10.1007/978-1-4614-4981-2_29
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-4980-5
Online ISBN: 978-1-4614-4981-2
eBook Packages: EngineeringEngineering (R0)