Abstract
Common approaches to control a data-center cooling system rely on approximated system/environment models that are built upon the knowledge of mechanical cooling and electrical and thermal management. These models are difficult to design and often lead to suboptimal or unstable performance. In this paper, we show how deep reinforcement learning techniques can be used to control the cooling system of a simulated data center. In contrast to common control algorithms, those based on reinforcement learning techniques can optimize a system’s performance automatically without the need of explicit model knowledge. Instead, only a reward signal needs to be designed. We evaluated the proposed algorithm on the open source simulation platform EnergyPlus. The experimental results indicate that we can achieve 22% improvement compared to a model-based control algorithm built into the EnergyPlus. To encourage the reproduction of our work as well as future research, we have also publicly released an open-source EnergyPlus wrapper interface (https://github.com/IBM/rl-testbed-for-energyplus) directly compatible with existing reinforcement learning frameworks.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Crawley, D.B., et al.: EnergyPlus: creating a new-generation building energy simulation program. Energy Build. 33(4), 319–331 (2001). https://doi.org/10.1016/S0378-7788(00)00114-6. Special Issue: BUILDING SIMULATION’99. http://www.sciencedirect.com/science/article/pii/S0378778800001146
Dhariwal, P., et al.: OpenAI Baselines (2017). https://github.com/openai/baselines
Heess, N., et al.: Emergence of locomotion behaviours in rich environments. arXiv (2017)
Inoue, T., Chaudhury, S., De Magistris, G., Dasgupta, S.: Transfer learning from synthetic to real images using variational autoencoders for precise position detection. In: IEEE International Conference on Image Processing (2018)
Inoue, T., De Magistris, G., Munawar, A., Yokoya, T., Tachibana, R.: Deep reinforcement learning for high precision assembly tasks. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2017)
Li, Y., Wen, Y., Guan, K., Tao, D.: Transforming cooling optimization for green data center via deep reinforcement learning. arXiv (2017). http://arxiv.org/abs/1709.05077
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529 (2015)
Munawar, A., et al.: MaestROB: a robotics framework for integrated orchestration of low-level control and high-level reasoning. In: IEEE International Conference on Robotics and Automation. IEEE (2018)
Pham, T.H., De Magistris, G., Tachibana, R.: OptLayer - practical constrained optimization for deep reinforcement learning in the real world. In: IEEE International Conference on Robotics and Automation (2018)
Report: data centre energy efficiency benchmarking - E2 Singapore (2015). http://www.e2singapore.gov.sg/DATA/0/docs/Resources/NEA%20DC%20Energy%20Benchmarking%20Summary-%20Final%20Report%20(3).pdf
Schulman, J., Levine, S., Abbeel, P., Jordan, M., Moritz, P.: Trust region policy optimization. In: Proceedings of International Conference on Machine Learning (2015)
Sun, J., Reddy, A.: Optimal control of building HVAC&R systems using complete simulation-based sequential quadratic programming (CSB-SQP). Build. Environ. 40(5), 657–669 (2005). https://doi.org/10.1016/j.buildenv.2004.08.011
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., Abbeel, P.: Domain randomization for transferring deep neural networks from simulation to the real world. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2017)
Wei, T., Wang, Y., Zhu, Q.: Deep reinforcement learning for building HVAC control. In: 2017 54th ACM/EDAC/IEEE Design Automation Conference (DAC), pp. 1–6, June 2017. https://doi.org/10.1145/3061639.3062224
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Moriyama, T., De Magistris, G., Tatsubori, M., Pham, TH., Munawar, A., Tachibana, R. (2018). Reinforcement Learning Testbed for Power-Consumption Optimization. In: Li, L., Hasegawa, K., Tanaka, S. (eds) Methods and Applications for Modeling and Simulation of Complex Systems. AsiaSim 2018. Communications in Computer and Information Science, vol 946. Springer, Singapore. https://doi.org/10.1007/978-981-13-2853-4_4
Download citation
DOI: https://doi.org/10.1007/978-981-13-2853-4_4
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-2852-7
Online ISBN: 978-981-13-2853-4
eBook Packages: Computer ScienceComputer Science (R0)