Towards Stochastic Constraint Programming: A Study of Onine Multi-Choice Knapsack with Deadlines

  • Thierry Benoist
  • Eric Bourreau
  • Yves Caseau
  • Benoît Rottembourg
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2239)


Constraint Programming (CP) is a very general programming paradigm that proved its efficiency on solving complex industrial problems. Most real-life problems are stochastic in nature, which is usually taken into account through different compromises, such as applying a deterministic algorithm to the average values of the input, or performing multiple runs of simulation. Our goal in this paper is to analyze different techniques taken either from practical CP applications or from stochastic optimization approaches. We propose a benchmark issued from our industrial experience, which may be described as an Online Multi-choice Knapsack with Deadlines. This benchmark is used to test a framework with four different dynamic strategies that utilize a different combination of the stochastic and combinatorial aspects of the problem. To evaluate the expected future state of the reservations at the time horizon, we either use simulation, average values, systematic study of the most probable scenarios, or yield management techniques.


Knapsack Problem Markov Decision Process Online Algorithm Average Gain Marginal Revenue 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [ABC+99]
    F. Afrati, E. Bampis, C. Chekuri, D. Karger, C. Kenyon, S. Khanna, I. Milis, M. Queyranne, M. Skutella, C. Stein, and M. Sviridenko: Approximation schemes for minimizing average weighted completion time with release dates. Proc. of the 1999 Symposium on Foundations of Computer Science (FOCS99), 1999.Google Scholar
  2. [AW96]
    A. N. Avramidis, J. R. Wilson: Integrated variance reduction strategies for simulation. Operations Research 44 (2): 327–346, 1996.zbMATHGoogle Scholar
  3. [B57]
    R. Bellman. Dynamic Programming. Princeton University Press, 1957.Google Scholar
  4. [BC89]
    D. P. Bertsekas, D. A. Castanon. Adaptative aggregation for infinite horizon dynamic programming. IEEE Transactions on Automatic Control, 34(6):589–598, 1989.zbMATHCrossRefMathSciNetGoogle Scholar
  5. [BDG95]
    C. Boutilier, R. Dearden, M. Goldszmidt: Exploiting structure in policy construction. In Proceedings of the 1995 international joint conference on artificial intelligence, 1995.Google Scholar
  6. [BL 97]
    J. Birge, F. Louveaux,: Introduction to Stochastic Programming Springer Series in Operations Research, 1997.Google Scholar
  7. [BT96]
    D. P. Bertsekas, J. N. Tsitsiklis: Neuro-dynamic Programming. Athena, Belmont, MA, 1996.Google Scholar
  8. [CSL99]
    Y. Caseau, G. Silverstein. F. Laburthe, A Meta-Heuristic Factory for Vehicle Routing Problems, Proc. of the 5th Int. Conference on Principles and Practice of Constraint Programming CP’99, LNCS 1713, Springer, 1999.Google Scholar
  9. [D63]
    F. D'Epenoux. A probabilistic production and inventory problem. Management Science: 10:98–108, 1963.CrossRefGoogle Scholar
  10. [DL95]
    T. Dean, S. H. Lin. Decomposition techniques for planning in stochastic domains, In Proceedings of the 1995 international joint conference on artificial intelligence, 1995.Google Scholar
  11. [DW00]
    A. Di Pierro, H. Wiklicky: Randomised Algorithms and Probabilistic Constraint Programming Proc. of the ERCIM/Compulog Workshop on Constraints, 19–21 June, Padova, Italy, 2000.Google Scholar
  12. [FW97]
    A. Fiat, G. J. Woeginger: Online Algorithms Lecture Notes in Computer Science, vol. 1442.Google Scholar
  13. [GR99]
    J. I. McGuill, G. J. Van Ryzin Revenue Management: research overview and prospects Transportation science vol.33 n°2, may 1999.Google Scholar
  14. [H60]
    R. A. Howard: Dynamic programming and markov Chains. MIT Press. Cambridge, 1960.Google Scholar
  15. [HG95]
    W. Harvey, M. Ginsberg: Limited Discrepancy Search. Proceedings of the 14th IJCAI, p. 607–615, Morgan Kaufmann, 1995.Google Scholar
  16. [HSS+96]
    L. Hall, A. S. Schulz, D. Shmoys, J. Wein: Scheduling to Minimize Average Completion Time: Off-line and On-line Approximation Algorithms. Proc of SODA: ACM-SIAM Symposium on Discrete Algorithms, 1996.Google Scholar
  17. [KC74]
    H. J. Kushner, C. H. Chen: Decomposition of systems governed by Markov chains. IEEE transactions on Automatic Control, AC-19(5):501–507, 1974.CrossRefMathSciNetGoogle Scholar
  18. [MHK+98]
    N. Meuleau, M. Hauskrecht, K.-E. Kim, L. Peshkin, L. P. Kaelbling, T. dean, C. Boutilier. Solving Very Large Weakly Coupled Markov Decision Processes. American Association for Artificial Intelligence, 1998.Google Scholar
  19. [MT90]
    S. Martello, P. Toth, Knapsack problems. Algorithms and computer implementations. John Wiley and Sons, West Sussex, England, 1990.Google Scholar
  20. [P98]
    R. Parr. Flexible Decompostion Algorithms for weakly coupled Markov Decision Problems. Uncertainty in Artificial Intelligence. Madison, Wisconsin, USA, 1998Google Scholar
  21. [P94]
    M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York, 1994.Google Scholar
  22. [PRK95]
    J. Papastavrou, S. Rajagopalan, A. Kleywegt: The Dynamic and Stochastic Knapsack Problem with Deadlines. Technical report, School of Industrial Engineering, Purdue University, West Lafayette, April 1995.Google Scholar
  23. [SC98]
    S. P. Singh, D. Cohn. How to dynamically merge Markov Decision Processes. In M. Mozer, M. Jordan and T. Petsche eds, NIPS-11. MIT Press, Cambridge, 1998.Google Scholar
  24. [WB92]
    L. R. Weatherford, S. E. Bodily: A Taxonomy and Research Overview of Perishable-Asset Revenue Managment: Yield Managment, Overbooking and Pricing. Operations Research 40:831–844, 1992Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Thierry Benoist
    • 1
  • Eric Bourreau
    • 1
  • Yves Caseau
    • 1
  • Benoît Rottembourg
    • 1
  1. 1.Bouygues e-labSt Quentin en Yvelines CedexFrance

Personalised recommendations