Pattern Learning and Decision Making in a Photovoltaic System
We study the effects of different decision making schemes on the accumulative rewards when photovoltaic (PV) facilities are intended as a potential replacement for conventional peaking power plants. As the amount of solar irradiance usable by a PV module follows a stochastic process, we compare the outcomes using the following two strategies in a stochastic environment: (1) employing an optimal decision making approach without any specific knowledge of the environment; and (2) optimal decision making based upon learning patterns of the environment process. We examine the possibility of integrating a pattern learning approach – called an ε-Machine – with a Partially Observable Markov Decision Process (POMDP). This approach has been motivated in part by the fact that efforts in extending traditional learning approaches to POMDPs have so far achieved only limited success. The PV facility in our model consists of a PV panel and a battery, with an associated local, non-critical load. Under the assumption that any PV generated power exceeding the maximum local consumption capacity must be dumped when the battery is full, the goal of the autonomous control agent is to maintain the maximum output potential to most effectively offset unexpected demand peaks, while minimizing energy wastage in the presence of strong solar irradiance. The environment is assumed to follow a Markov process of a different order than the part of the system under the influence of the agent.
KeywordsBelief State Causal State Maximum Power Point Tracker Pattern Learn Partially Observable Markov Decision Process
Unable to display preview. Download preview PDF.
- 2.Littman, M.L., Dean, T.L., Kaelbling, L.P.: On the complexity of solving Markov Decision Problems. In: Proceedings of the Eleventh Annual Conference on Uncertainty in Artificial Intelligence (UAI 1995), Montreal, Québec, Canada, pp. 394–402 (1995)Google Scholar
- 3.Shalizi, C.R., Shalizi, K.L., Crutchfield, J.P.: An algorithm for pattern discovery in time series. Journal of Machine Learning Research (to appear)Google Scholar
- 4.Shalizi, C.R.: Causal Architecture, Complexity and Self-Organization for Time Series and Cellular Automatas. PhD thesis, University of Wisconsin at Madison (2001)Google Scholar
- 6.Kim, J., Torregoza, J.P.M., Kong, I., Hwang, W.: Photovoltaic cell battery model for wireless sensor networks. International Journal of Computer Science and Network Security 6(9B) (2006)Google Scholar
- 7.Chiasserini, C., Rao, R.: A model for battery pulsed discharge with recovery effect. In: Wireless Communications and Networking Conference, pp. 636–639 (1999)Google Scholar
- 8.Rao, V., Singhal, G., Kumar, A., Navet, N.: Battery model for embedded systems. In: 18th International Conference on VLSI Design, pp. 57–63 (2005)Google Scholar
- 9.Murphy, K.: A survey of POMDP solution techniques. Technical report, U.C. Berkeley (2000)Google Scholar
- 10.Meuleau, N., Kim, K.E., Kaelbling, L.P., Cassandra, A.R.: Solving POMDPs by searching the space of finite policies. In: Proceedings of the Fifteenth Conf. on Uncertainty in Artificial Intelligence, pp. 417–426 (1999)Google Scholar
- 11.Roy, N.: Finding Approximate POMDP Solutions Through Belief Compression. PhD thesis, Carnegie Mellon University (2003)Google Scholar