Reinforcement Learning Based on the Bayesian Theorem for Electricity Markets Decision Support

  • Tiago M. SousaEmail author
  • Tiago Pinto
  • Isabel Praça
  • Zita Vale
  • Hugo Morais
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 290)


This paper presents the applicability of a reinforcement learning algorithm based on the application of the Bayesian theorem of probability. The proposed reinforcement learning algorithm is an advantageous and indispensable tool for ALBidS (Adaptive Learning strategic Bidding System), a multi-agent system that has the purpose of providing decision support to electricity market negotiating players. ALBidS uses a set of different strategies for providing decision support to market players. These strategies are used accordingly to their probability of success for each different context. The approach proposed in this paper uses a Bayesian network for deciding the most probably successful action at each time, depending on past events. The performance of the proposed methodology is tested using electricity market simulations in MASCEM (Multi-Agent Simulator of Competitive Electricity Markets). MASCEM provides the means for simulating a real electricity market environment, based on real data from real electricity market operators.


Bayesian Network Success Probability Multiagent System Electricity Market Dynamic Bayesian Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Pinto, T., et al.: Multiagent System for Adaptive Strategy Formulation in Electricity Markets. In: IEEE Symposium on Intelligent Agents, SSCI-IA. IEEE Symposium Series on Computational Intelligence (2011)Google Scholar
  2. 2.
    Dore, A., Regazzoni, C.: Interaction Analysis with a Bayesian Trajectory Model. IEEE Intelligent Systems 25(3), 32–40 (2010)CrossRefGoogle Scholar
  3. 3.
    Korb, K., Nicholson, A.: Bayesian Artificial Intelligence. Chapman & Hall/CRC (2003)Google Scholar
  4. 4.
    Shahidehpour, M., et al.: Market Operations in Electric Power Systems: Forecasting, Scheduling, and Risk Management, pp. 233–274. Wiley-IEEE Press (2002)Google Scholar
  5. 5.
    Praça, I., et al.: MASCEM: A Multi-Agent System that Simulates Competitive Electricity Markets. IEEE Intelligent Systems 18(6), 54–60 (2003)CrossRefGoogle Scholar
  6. 6.
    Vale, Z., et al.: MASCEM: Electricity Markets Simulation with Strategic Agents. IEEE Intelligent Systems 26(2), 9–17 (2011)CrossRefMathSciNetGoogle Scholar
  7. 7.
    Pinto, T., et al.: A new approach for multi-agent coalition formation and management in the scope of electricity markets. Energy 36(8), 5004–5015 (2011)CrossRefGoogle Scholar
  8. 8.
    Pinto, T., Vale, Z., Rodrigues, F., Morais, H., Praça, I.: Strategic Bidding Methodology for Electricity Markets using Adaptive Learning. In: Mehrotra, K.G., Mohan, C.K., Oh, J.C., Varshney, P.K., Ali, M. (eds.) IEA/AIE 2011, Part II. LNCS (LNAI), vol. 6704, pp. 490–500. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  9. 9.
    Sun, J., Tesfatsion, L.: Dynamic Testing of Wholesale Power Market Designs: An Open-Source Agent-Based Framework. Computational Economics 30(3) (2007)Google Scholar
  10. 10.
    Amjady, N., et al.: Day-ahead electricity price forecasting by modified relief algorithm and hybrid neural network. IET Generation, Transmission & Distribution 4(3), 432–444 (2010)CrossRefGoogle Scholar
  11. 11.
    Cao, L., Gorodetsky, V., Mitkas, P.: Agent Mining: The Synergy of Agents and Data Mining. IEEE Intelligent Systems 24(3), 64–72 (2009)CrossRefGoogle Scholar
  12. 12.
    Bompard, E., et al.: A game theory simulator for assessing the performances of competitive electricity markets. IEEE St. Petersburg Power Tech (2005)Google Scholar
  13. 13.
    MIBEL - Operador del Mercado Ibérico de Energia – Iberian Electricity Market Operator (accessed on December 2013),

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Tiago M. Sousa
    • 1
    Email author
  • Tiago Pinto
    • 1
  • Isabel Praça
    • 1
  • Zita Vale
    • 1
  • Hugo Morais
    • 2
  1. 1.GECAD – Knowledge Engineering and Decision-Support Research Center, Institute of EngineeringPolitechnic of Porto (ISEP/IPP)PortoPortugal
  2. 2.Automation and Control GroupTechnical University of DenmarkCopenhagenDenmark

Personalised recommendations