Recommendations as a Game: Reinforcement Learning for Recommendation Engines

  • Alexander Paprotny
  • Michael Thess
Part of the Applied and Numerical Harmonic Analysis book series (ANHA)


We describe the application of reinforcement learning to recommendation engines. At this, we introduce RE-specific empirical assumptions to reduce the complexity of RL in order to make it applicable to real-live recommendation problems. Especially, we provide a new approach for estimating transition probabilities of multiple recommendations based on that of single recommendations. The estimation of transition probabilities for single recommendations is left as an open problem that is covered in Chap.  5. Finally, we introduce a simple framework for testing online recommendations.


Recommendation Engine Reinforcement Learning Theory Multiple Recommendations Single Recommendation Estimated Transition Probabilities 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. [GR04]
    Golovin, N., Rahm, E.: Reinforcement Learning Architecture for Web Recommendations. Proc. ITCC2004, IEEE (2004)Google Scholar
  2. [Mah10]
    Mahmood, T.: Learning User-Adapted Strategies in Conversational Recommender Systems: Application of Reinforcement Learning to E-commerce Portals for Learning a System Behavior that is Adapted to the Users in an Interaction Context. VDM Verlag Dr. Müller, Saarbrücken (2010)Google Scholar
  3. [RSP05]
    Rojanavasu, P., Srinil, P., Pinngern, O.: New recommendation system using reinforcement learning. Proceedings of the Fourth International Conference on eBusiness, Bangkok, 19–20 Nov 2005Google Scholar
  4. [SHB05]
    Shani, G., Heckerman, D., Brafman, R.I.: An MDP-based recommender system. J. Mach. Learn. Res. 6, 1265–1295 (2005)zbMATHMathSciNetGoogle Scholar
  5. [SKKR00]
    Sarwar, B., Karypis, G., Konstan, J., Riedl J.: Analysis of recommendation algorithms for e-commerce. EC’00, Minneapolis, 17–20 Oct 2000Google Scholar
  6. [TGK07]
    Taghipour, N., Ghidary, S.S., Kardan A.: Using q-learning for web recommendations from web usage data. In: 12th International CSI Computer Conference, Teheran (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Alexander Paprotny
    • 1
  • Michael Thess
    • 2
  1. 1.Research and Developmentprudsys AGBerlinGermany
  2. 2.Research and Developmentprudsys AGChemnitzGermany

Personalised recommendations