Reinforced Learning in Market Games

  • Edward W. Piotrowski
  • Jan Sładkowski
  • Anna Szczypińska
Part of the New Economic Windows book series (NEW)


Financial markets investors are involved in many games — they must interact with other agents to achieve their goals. Among them are those directly connected with their activity on markets but one cannot neglect other aspects that influence human decisions and their performance as investors. Distinguishing all subgames is usually beyond hope and resource consuming. In this paper we study how investors facing many different games, gather information and form their decision despite being unaware of the complete structure of the game. To this end we apply reinforcement learning methods to the Information Theory Model of Markets (ITMM). Following Mengel, we can try to distinguish a class Γ of games and possible actions (strategies) \(a_{m_i }^i\) for i-th agent. Any agent divides the whole class of games into analogy subclasses she/he thinks are analogous and therefore adopts the same strategy for a given subclass. The criteria for partitioning are based on profit and costs analysis. The analogy classes and strategies are updated at various stages through the process of learning. We will study the asymptotic behavior of the process and attempt to identify its crucial stages, e.g., existence of possible fixed points or optimal strategies. Although we focus more on the instrumental aspects of agents behaviors, various algorithm can be put forward and used for automatic investment. This line of research can be continued in various directions.


Reinforcement Learning Market Game Perfect Competition Bias Ratio Brute Force Approach 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cassandras, C. G., Lafortune, S., Introduction to Discrete Event Systems, volume 11 of The Kluwer International Series in Discrete Event Dynamic Systems, Kluwer Academic Publisher, Boston, MA, (1999)CrossRefGoogle Scholar
  2. 2.
    Taleb N. N., The Black Swan: The Impact of the Highly Improbable, Random House, Inc (2008)Google Scholar
  3. 3.
    Crutchfield, J. P., Shalizi, C. R., Physical Review E 59 (1999) 275CrossRefGoogle Scholar
  4. 4.
    Benveniste, A., Metevier, M., Priouret, P., Adaptive Algorithms and Stochastic Approximation, Berlin: Springer Verlag (1990)CrossRefGoogle Scholar
  5. 5.
    Fudenberg, D., Levine D. K., The Theory of Learning in Games, Cambridge: MIT-Press (1998)Google Scholar
  6. 6.
    Kuschner, H. J., Lin G. G., Stochastic Approximation and Recursive Algorithms and Applications, New York: Springer (2003)Google Scholar
  7. 7.
    Watkins, C. J. C. H., Dayan, P., Q-learning, Machine Learning 8 (1992) 279Google Scholar
  8. 8.
    Farias, V. F., Moallemi, C. C., Weissman, T., Van Roy, B., Universal Reinforcement Learning, arXiv:0707.3087v1 [cs.IT]Google Scholar
  9. 9.
    Shneyerov, A., Chi Leung Wong, A., The rate of convergence to a perfect competition of a simple Matching and bargaining Mechanism, Univ of Britich Columbia preprint (2007)Google Scholar
  10. 10.
    Littman, M. L., Markov games as a framework for multi-agent reinforcement learning, Brown Univ. preprintGoogle Scholar
  11. 11.
    Fliess, M., Join, C., A mathematical proof of the existence of trends in financial time series, arXiv:0901.1945v1 [q-fin.ST]Google Scholar
  12. 12.
    Kelly, J. L., Jr., A New Interpretation of Information Rate, The Bell System Technical Journal 35 (1956) 917CrossRefGoogle Scholar
  13. 13.
    Thorpe, E. O., Optimal gambling systems for favorable games, Revue de l’Institut International de Statistique / Review of the International Statistical Institute, 37 (1969) 273CrossRefGoogle Scholar
  14. 14.
    Piotrowski, E. W., Schroeder, M., Kelly Criterion revisited: optimal bets, Eur. Phys. J. B 57 (2007) 201CrossRefGoogle Scholar
  15. 15.
    Piotrowski, E.W., łuczka, J., The relativistic velocity addition law optimizes a forecast gambler’s profit, submmited to Physica A; arXiv:0709.4137v1 []Google Scholar
  16. 16.
    Piotrowski, E. W., Sładkowski, J., The Merchandising Mathematician Model, Physica A 318 (2003) 496CrossRefGoogle Scholar
  17. 17.
    Mengel, F., Learning across games, Univ. of Alicante report WP-AD 2007-05Google Scholar
  18. 18.
    Piotrowski, E. W., Sładkowski, J., Arbitrage risk induced by transaction costs, Physica A 331 (2004) 233CrossRefGoogle Scholar
  19. 19.
    Piotrowski, E. W., Fixed point theorem for simple quantum strategies in quantum market games, Physica A 324 (20043) 196Google Scholar
  20. 20.
    Piotrowski, E. W., Sładkowski, J., Quantum computer: an appliance for playing market games, International Journal of Quantum Information 2 (2004) 495CrossRefGoogle Scholar
  21. 21.
    Miakisz, K., Piotrowski, E. W., Sładkowski, J., Quantization of Games: Towards Quantum Artificial Intelligence, Theoretical Computer Science 358 (2006) 15CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Italia 2010

Authors and Affiliations

  • Edward W. Piotrowski
    • 1
  • Jan Sładkowski
    • 2
  • Anna Szczypińska
    • 3
  1. 1.Institute of MathematicsUniversity of BiałystokBiałystokPoland
  2. 2.Institute of PhysicsUniversity of SilesiaKatowicePoland
  3. 3.Institute of PhysicsUniversity of SilesiaKatowicePoland

Personalised recommendations