Optimal Threshold Policies for Robust Data Center Control

  • Paul Weng
  • Zeqi Qiu (邱泽麒)
  • John Costanzo
  • Xiaoqi Yin (阴小骐)
  • Bruno Sinopoli


With the simultaneous rise of energy costs and demand for cloud computing, efficient control of data centers becomes crucial. In the data center control problem, one needs to plan at every time step how many servers to switch on or off in order to meet stochastic job arrivals while trying to minimize electricity consumption. This problem becomes particularly challenging when servers can be of various types and jobs from different classes can only be served by certain types of server, as it is often the case in real data centers. We model this problem as a robust Markov decision process (i.e., the transition function is not assumed to be known precisely). We give sufficient conditions (which seem to be reasonable and satisfied in practice) guaranteeing that an optimal threshold policy exists. This property can then be exploited in the design of an efficient solving method, which we provide. Finally, we present some experimental results demonstrating the practicability of our approach and compare with a previous related approach based on model predictive control.

Key words

data center control Markov decision process threshold policy robustness 

CLC number



Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    KAPLAN J M, FOREST W, KINDLER N. Revolutionizing data center energy efficiency [R]. Chicago: McKinsey & Company, 2008.Google Scholar
  2. [2]
    MASTROLEON L, BAMBOS N, KOZYRAKIS C, et al. Automatic power management schemes for Internet servers and data centers [C]//Global Telecommunications Conference, 2005. Missouri: IEEE, 2005: 1–5.Google Scholar
  3. [3]
    PAROLINI L, SINOPOLI B, KROGH B H. Reducing data center energy consumption via coordinated cooling and load management [C]//Proceedings of the Conference on Power Aware Computing and Systems. [s.l.]: USENIX Association, 2008: 1–5.Google Scholar
  4. [4]
    PAROLINI L, SINOPOLI B, KROGH B H, et al. A cyber-physical systems approach to data center modeling and control for energy efficiency [J]. Proceedings of the IEEE, 2011, 100(1): 254–268.CrossRefGoogle Scholar
  5. [5]
    YIN X, SINOPOLI B. Adaptive robust optimization for coordinated capacity and load control in data centers [C]// the International Conference on Decision and Control. [s.l.]: IEEE, 2014: 5674–5679.Google Scholar
  6. [6]
    FEINBERG E, ZHANG X. Optimizing cloud utilization via switching decisions [J]. Performance Evaluation Review, 2014, 41(4): 57–60.CrossRefGoogle Scholar
  7. [7]
    LUBIN B, KEPHART J O, DAS R, et al. Expressive power-based resource allocation for data centers [C]//International Joint Conference on Artificial Intelligence. Pasadena, California: Morgan Kaufmann Publishers Inc., 2009: 1451–1456.Google Scholar
  8. [8]
    BOD’IK P. Automating datacenter operations using machine learning [D]. Berkeley: University of California, Berkeley, 2010.Google Scholar
  9. [9]
    GAO J. Machine learning applications for data center optimization [EB/OL]. (2014-10-27) [2017-09-25]. Scholar
  10. [10]
    GHASEMI M, LUBIN B. Modeling multi-attribute demand for sustainable cloud computing with copulae [C]//International Conference on Artificial Intelligence. Buenos Aires, Argentina: AAAI Press, 2015: 2596–2602.Google Scholar
  11. [11]
    HORDIJK A, SCHOUTEN F V D D. On the optimality of (s, S)-policies in continuous review inventory models [J]. SIAM Journal on Applied Mathematics, 1986, 46(5): 912–929.MathSciNetCrossRefMATHGoogle Scholar
  12. [12]
    KALIN D. On the optimality of (_, s)-policies [J]. Mathematics of Operations Research, 1980, 5(2): 293–307.MathSciNetCrossRefMATHGoogle Scholar
  13. [13]
    HYON E, JEAN-MARIE A. Scheduling services in a queueing system with impatience and setup costs [J]. The Computer Journal, 2012, 55(5): 553–563.CrossRefGoogle Scholar
  14. [14]
    FOX M, LONG D, MAGAZZENI D. Automatic construction of efficient multiple battery usage policies [C]//International Joint Conference on Artificial Intelligence. Freiburg: AAAI Press, 2011: 2620–2625.Google Scholar
  15. [15]
    PETRIK M, WU X. Optimal threshold control for energy arbitrage with degradable battery storage [C]//Conference on Uncertainty in Artificial Intelligence. Amsterdam: AUAI Press, 2015: 692–701.Google Scholar
  16. [16]
    ERSEGHE T, ZANELLA A, CODEMO C. Markov decision processes with threshold based piecewise linear optimal policies [J]. IEEE Wireless Communication Letters, 2013, 2(4): 459–462.CrossRefGoogle Scholar
  17. [17]
    KOOLE G. Monotonicity in Markov reward and decision chains: Theory and applications [J]. Foundations & Trends® in Stochastic Systems, 2007, 1(1):1–76.MathSciNetCrossRefMATHGoogle Scholar
  18. [18]
    VEINOTT A. Optimal policy for a multi-product, dynamic, nonstationary inventory problem [J]. Management science, 1965, 12(3): 206–222.MathSciNetCrossRefMATHGoogle Scholar
  19. [19]
    JUAN D C, LI L, PENG H K, et al. Beyond Poisson: Modeling inter-arrival time of requests in a datacenter [C]//Pacific-Asia Conference on Knowledge Discovery and Data Mining. China, Taiwan: Springer, 2014: 198–209.CrossRefGoogle Scholar
  20. [20]
    GIVAN R, LEACH S M, DEAN T. Bounded parameter Markov decision processes [J]. Artificial Intelligence, 2000, 122(1/2): 71–109.MathSciNetCrossRefMATHGoogle Scholar
  21. [21]
    NILIM A, GHAOUI L E. Robustness in Markov decision problems with uncertain transition matrices [C]//Advances in Neural Information Processing Systems 16 (NIPS 2003). Vancouver and Whistler, British Columbia: NIPS, 2003: 1–8.Google Scholar
  22. [22]
    PUTERMAN M L. Markov decision processes: Discrete stochastic dynamic programming [M]. America: John Wiley & Sons, 1994: 1–353.MATHGoogle Scholar
  23. [23]
    LITTMAN M L, DEAN T L, KAELBLING L P. On the complexity of solving Markov decision problems [C]//11th Conference on Uncertainty in Artificial Intelligence. San Francisco: Morgan Kaufmann Publishers, 1995: 394–402.Google Scholar
  24. [24]
    Index of /other/pageviews/2015/ [EB/OL]. (2016-12-30) [2017-09-25]. pageviews/2015/Google Scholar
  25. [25]
    SUROVIK D A, SCHEERES D J. Heuristic search and receding-horizon planning in complex spacecraft orbit domains [C]//Proceeding of the 25th International Joint Conference on Artificial Intelligence. Jerusalem, Israel: AAAI Press, 2015: 291–295.Google Scholar
  26. [26]
    WIERING M, OTTERLO M V. Reinforcement Learning: State-of-the-Art [M]. Germany: Springer Publishing Company, Incorporated, 2012: 1–650.CrossRefGoogle Scholar
  27. [27]
    HADOUX E, BEYNIER A, WENG P. Solving hidden-semi-Markov-mode Markov decision problems [C]//International Conference on Scalable Uncertainty Management. New York: Springer-Verlag. 2014: 176–189.Google Scholar

Copyright information

© Shanghai Jiaotong University and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Paul Weng
    • 1
    • 2
  • Zeqi Qiu (邱泽麒)
    • 3
  • John Costanzo
    • 3
  • Xiaoqi Yin (阴小骐)
    • 3
  • Bruno Sinopoli
    • 3
  1. 1.SYSU-CMU Joint Institute of Engineering, School of Electronics and Information TechnologySun Yat-sen UniversityGuangzhouChina
  2. 2.SYSU-CMU Joint Research InstituteGuangdongChina
  3. 3.Department of Electrical & Computer EngineeringCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations