Dynamic Pricing of Information Products Based on Reinforcement Learning: A Yield-Management Approach

  • Michael Schwind
  • Oliver Wendt
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2479)


Pricing of information services gains an increasing importance in an IT environment, which is characterized by more and more decentralized computing resources (e.g. P-2-P computing). Even if pricing theory represents a kernel domain of economic research the pricing problem related to automated information production processes could not be handled satisfactory. This stems from the combination of high fixed costs with negligible variable costs. Especially in airline industries this problem is addressed by heuristics in the so called “ Yield Management” (YM) domain. The paper presented here, shows the transferability of these methods to the information production and services domain. Pricing a bundle of complementary resources can not be solved by the simple addition of value functions. Therefore we introduce Machine Learning (ML) techniques to master complexity. Artificial Neural Networks (ANN) are used for the joint representation of the multidimensional value functions and Genetic Algorithms (GA) should help train them in a first effort. While this does not lead to outstanding results, we try Reinforcement Learning (RL) in a second approach. This ML method provides encouraging results for efficient adaptive pricing of resource attribution related to the multidimensional YM problem.


Reinforcement Learn Dynamic Price Stochastic Dynamic Program Policy Iteration Price Theory 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alstrup, J., Boas, S., Madsen, O. et al.: Booking Policy for Flights with two Types of Passengers. European Journal of Operational Research 27 (1986) 274–288CrossRefGoogle Scholar
  2. 2.
    Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957)Google Scholar
  3. 3.
    Belobaba, P.: Application of a Probabilistic Decision Model to Airline Seat Inventory-Control. Operations Research 37 (1989) 183–197Google Scholar
  4. 4.
    Bertsch, L.: Expertengestützte Dienstleistungskostenrechnung. Poeschel, Stuttgart (1990)Google Scholar
  5. 5.
    Bertsekas, D.; Tsitsiklis, J.: Neuro-Dynamic Programming. Athena Scientific, Belmont MA (1996)zbMATHGoogle Scholar
  6. 6.
    Bertsimas, D.; Popescu, I.: Revenue Management in a Dynamic Network Environment, Submitted to Transportation Science (2000)Google Scholar
  7. 7.
    Bishop, C.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)Google Scholar
  8. 8.
    Chen, V. C, Günther, D., Johnson, E.: Airline Yield Management: Optimal Bid Prices for Single Hub Problems without Cancellations, submitted to Journal of Transportation Science (1999)Google Scholar
  9. 9.
    Debreu, G.: Theory of Value. John Wiley and Sons, New York (1959)zbMATHGoogle Scholar
  10. 10.
    Doerninger, W.: Approximating General Markovian Decision Problems by Clustering their State-and Action-Spaces; Math. Operationsforschung und Statistik; Ser. Optimization 15, (1984) 135–144Google Scholar
  11. 11.
    Goldberg, D. E.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison Wesley, Reading MA (1989)zbMATHGoogle Scholar
  12. 12.
    Hildenbrand, W., Kirman A. P.: Introduction to Equilibrium Analysis, Amsterdam (North Holland) (1976)zbMATHGoogle Scholar
  13. 13.
    Holland, J. H.: Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and AI. University of Michigan Press, Ann Arbor MI (1975)Google Scholar
  14. 14.
    Hornick, S.: Value Based Revenue Management-A New Paradigm for Airline Seat Inventory Control. In Behrendt, R.; Bertsch, L. H., (ed.) Advanced Software Technology in Air Transport 91, AIT-Verlag, Halbergmoos (1991)Google Scholar
  15. 15.
    Humair, S.: Yield Management for Telecommunication Networks: Defining a New Landscape. Dissertation, Massachusetts Institute of Technology, Cambridge MA (2001)Google Scholar
  16. 16.
    Kimes, S.: Yield Management. A Tool for Capacity Constrained Firms. Journal of Operations Management 4 (1989) 348–363CrossRefGoogle Scholar
  17. 17.
    Kohonen, T.: Self-organized Formation of Topologically Correct Feature Maps. In: Biological Cybernetics 43 (1982) 59–69zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Kohonen, T.: Analysis of a Simple Self-Organizing Process. In: Biological Cybernetics 44 (1982) 135–140zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Kohonen, T.: Self-Organizing Maps. Springer Series in Information Sciences, Springer, Berlin (1995)Google Scholar
  20. 20.
    Lloyd S.: Least squares quantization in PCM. IEEE Trans. Inform. Theory, vol. IT-28, (1982) 2MathSciNetGoogle Scholar
  21. 21.
    MacQueen, J.: Some methods for classification an analysis of multivariate observations. In: LeCam, L.; Neyman, J. (eds.): Proc. of the 5 th Berkeley Symposium on Mathematical Statistics, and Probability 1, University of California Press, Berkeley CA (1967) 281–297Google Scholar
  22. 22.
    Malinvaud, E.: Lectures on Microeconomic Theory. Amsterdam (North Holland), 4 th ed. (1974)zbMATHGoogle Scholar
  23. 23.
    Martinez, T. M.; Schulten, K. S.: A “Neural-Gas” Network Learns Topologies. In: Kohonen, Teuvo.; Mäkisara, K.; Simula, O.; Kangas, J. (eds.): Artificial Neural Networks, North Holland, Amsterdam (1991) 397–402Google Scholar
  24. 24.
    Martinez, T. M.: Competitive Hebbian Learning Rule Forms Perfectly Topology Preserving Maps. In: ICANN’ 93: International Conference on Neural Networks, Springer, Amsterdam (1993) 427–434Google Scholar
  25. 25.
    Nieschlag, R., Dichtel E., Hörschgen H.: Marketing. 17. Auflage, Duncker & Humboldt, Berlin (1994)Google Scholar
  26. 26.
    Nollau V., Hahnewald-Busch A.: An Approximation Procedure for Stochastic Dynamic Programming in Countable State Spaces; Math. Operationsforschung und Statistik; Ser. Opt. 9 (1978) 109–117zbMATHMathSciNetGoogle Scholar
  27. 27.
    Puterman, M.: Markov Decision Problems. Wiley, New York (1994)Google Scholar
  28. 28.
    Remmers, J.: Yield Management im Tourismus. In: Schertler (ed.) Tourismus als Informationsgeschäft. Ueberreuter, Wien (1994)Google Scholar
  29. 29.
    Ritter, H.; Martinetz, T.; Schulten, K.: Neuronale Netze: Eine Einführung in die Neuroinformatik selbst organisierender Netzwerke. Addison-Wesley, Reading MA (1990)Google Scholar
  30. 30.
    Schwefel, Hans-Paul: Evolution and Optimum Seeking. Wiley-Interscience, New York (1995)Google Scholar
  31. 31.
    Simon, H.: Preismanagement. 2. Auflage, Gabler, Wiesbaden (1992)Google Scholar
  32. 32.
    Smith, B., Leimkuhler, J., Darrow, R.: Yield Management at American Airlines. Interfaces 1 (1992) 8–31CrossRefGoogle Scholar
  33. 33.
    Sutton, R.; Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge MA (1999) 52Google Scholar
  34. 34.
    Talluri, K., Ryzin, G.: An Analysis of Bid-Price Controls for Network Revenue Management. Working Paper of the Management Science and Operations Management Division, Columbia Business School, Columbia University New York (1996)Google Scholar
  35. 35.
    Vogel, H.: Yield Management-Optimale Kapazität für jedes Marktsegment zum richtigen Preis. Fremdenverkehrswirtschaft International 22 (1998)Google Scholar
  36. 36.
    Wetherford, L., Bodily S.: A Taxonomy and Research Overview of Perishable-Asset Revenue Management: Yield Management, Overbooking and Pricing. Operations Research 40 (1992) 831–844Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Michael Schwind
    • 1
  • Oliver Wendt
    • 1
  1. 1.Information SystemsFrankfurt UniversityFrankfurtGermany

Personalised recommendations