Representation of Learning and Adaptation
The agent-based electricity sector simulation model that will be fully described in Chap. 5 comprises adaptive agents that bid in interrelated electricity and CO2 allowance markets. In this model, the representation of strategic bidding behavior is a central element, and is realized through learning algorithms.
The review of existing agent-based approaches to simulating electricity markets (see Sect. 3.3) revealed that numerous different models of adaptive and learning behavior have been applied by ACE researchers, most of them without clear justification, let alone comparison of several alternatives that determine the most appropriate model or the best parameter combination. This observation can be made for models involving learning in other economic contexts, too (Brenner, 2006). Meanwhile, the representation of learning constitutes the core element of agent-based simulations and much effort should be spent for justifying the chosen approach. If several models of learning or adaptation are suitable, they may all be used, thereby possibly rejecting those conclusions that are not confirmed by all applied learning models, thus making confirmed simulation results better grounded.
Following the aforementioned argument, this chapter reviews the candidates for behavioral representations in agent-based electricity market models. Three basic concepts of behavioral models have attracted attention by economic researchers (Duffy, 2006): zero-intelligence approaches, reinforcement and belief-based models, and evolutionary concepts. The last two learning models will be presented in Sects. 4.1 and 4.2. This presentation is followed by the description of a set of simulation runs that have been carried out in order to decide which learning algorithm best represents agents’ bidding behavior in a simplified electricity market scenario (Sect. 4.3). A number of different variants and parameter combinations of three learning models have been tested and compared extensively. They can be categorized as reinforcement learning (RL) and hybrid reinforcement and belief-based learning algorithms.
KeywordsReinforcement Learning Learning Model Electricity Market Bidding Behavior Reinforcement Learning Algorithm
Unable to display preview. Download preview PDF.