Reinforcement Learning Techniques for the Control of WasteWater Treatment Plants
Since water pollution is one of the most serious environmental problems today, control of wastewater treatment plants (WWTPs) is a crucial issue nowadays and stricter standards for the operation of WWTPs have been imposed by authorities. One of the main problems in the automation of the control of Wastewater Treatment Plants (WWTPs) appears when the control system does not respond as it should because of changes on influent load or flow. Thus, it is desirable the development of autonomous systems that learn from interaction with a WWTP and that can operate taking into account changing environmental circumstances. In this paper we present an intelligent agent using reinforcement learning for the oxygen control in the N-Ammonia removal process in the well known Benchmark Simulation Model no.1 (BSM1). The aim of the approach presented in this paper is to minimize the operation cost changing the set-points of the control system autonomously.
KeywordsExpert System Wastewater Treatment Plant Reinforcement Learning Markov Decision Process Fuzzy Neural Network
Unable to display preview. Download preview PDF.
- 1.Alex, J., Benedetti, L., Copp, J., Gernaey, K.V., Jeppsson, U., Nopens, I., Pons, M.N., Rosen, C., Steyer, J.P., Vanrolleghem, P.: Benchmark Simulation Model no. 1 (BSM1). In: IWA Taskgroup on Benchmarking of Control Strategies for WWTP, p. 61 (April 2008)Google Scholar
- 3.Bertsekas, D.P.: Dynamic Programming and Optimal Control, 3rd edn., vol. 2. Athenas Scientific (2007)Google Scholar
- 7.Copp, J.: The COST Simulation Benchmark: Description and Simulator Manual, p. 154, Office for Official Publications of the European Community, Luxembourg (2002); ISBN: 92-894-1658-0Google Scholar
- 11.Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)Google Scholar
- 13.Olsson, G., Nielsen, M., Yuan, Z., Lynggaard-Jensen, A., Steyer, J.P.: Instrumentation, Control and Automation in Wastewater Systems. IWA Publishing (2005)Google Scholar
- 16.Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar