Intelligent Pump Scheduling Optimization in Water Distribution Networks

  • Antonio CandelieriEmail author
  • Riccardo Perego
  • Francesco Archetti
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11353)


In this paper, the authors are concerned with Pump Scheduling Optimization in Water Distribution Networks, targeted on the minimization of the energy costs subject to operational constraints, such as satisfying demand, keeping pressures within certain bounds to reduce leakage and the risk of pipe burst, and keeping reservoir levels within bounds to avoid overflow. Urban water networks are generating huge amounts of data from flow/pressure sensors and smart metering of household consumption. Traditional optimization strategies fail to capture the value hidden in real time data assets. In this paper the authors are proposing a sequential optimization method based on Approximate Dynamic Programming in order to find a control policy defined as a mapping from states of the system to actions, i.e. pump settings. Q-Learning, one of the Approximate Dynamic Programming algorithms, well known in the Reinforcement Learning community, is used. The key difference is that usual Mathematical Programming approaches, including stochastic optimization, requires knowing the water demand in advance or, at least, to have a reliable and accurate forecasting. On the contrary, Approximate Dynamic Programming provides a policy, that is a strategy to decide how to act time step to time step according to the observation of the physical system. Results on the Anytown benchmark network proved that the optimization policy/strategy identified through Approximate Dynamic Programming is robust with respect to modifications of the water demand and, therefore, able to deal with real time data without any distributional assumption.


Reinforcement learning Dynamic programming Pump scheduling optimization 


  1. 1.
    Bellman, R.E.: Dynamic Programming. Princeton University Press, Princeton (1957)zbMATHGoogle Scholar
  2. 2.
    Candelieri, A., Soldi, D., Archetti, F.: Short-term forecasting of hourly water consumption by using automatic metering readers data. Procedia Eng. 119, 844–853 (2015)CrossRefGoogle Scholar
  3. 3.
    Candelieri, A.: Clustering and Support Vector Regression for Water Demand Forecasting and Anomaly Detection. Water 9(3), 224 (2017)CrossRefGoogle Scholar
  4. 4.
    De Paola, F., Fontana, N., Giugni, M., Marini, G., Pugliese, F.: An application of the Harmony-Search Multi-Objective (HSMO) optimization algorithm for the solution of pump scheduling problem. Procedia Eng. 162, 494–502 (2016)CrossRefGoogle Scholar
  5. 5.
    Ertin, E., Dean, A.N., Moore, M.L., Priddy, K.L.: Dynamic optimization for optimal control of water distribution systems. Applications and science of computational intelligence IV. Proc. SPIE 4390, 142–149 (2001)CrossRefGoogle Scholar
  6. 6.
    Fracasso, P.T., Barnes, F.S., Costa, A.H.R.: Energy cost optimization in water distribution systems using Markov decision processes. In: International Green Computing Conference Proceedings, Arlington, pp. 1–6 (2013)Google Scholar
  7. 7.
    Fracasso, P.T., Barnes, F.S., Costa, A.H.R.: Optimized Control for Water Utilities. Procedia Eng. 70, 678–687 (2014)CrossRefGoogle Scholar
  8. 8.
    Howard, R.A.: Dynamic Programming and Markov Processes. MIT Press, Cambridge (1960)zbMATHGoogle Scholar
  9. 9.
    Ikonen, E., Bene, J.: Scheduling and disturbance control of a water distribution network. In: Proceedings of 18th World Congress of the International Federation of Automatic Control (IFAC 2011), Milano, Italy (2011)Google Scholar
  10. 10.
    Ikonen, E., Selek, I., Tervaskanto, M.: Short-term pump schedule optimization using MDP and neutral GA. IFAC Proc. Vol. 43(1), 315–320 (2010)CrossRefGoogle Scholar
  11. 11.
    Mala-Jetmarova, H., Sultanova, N., Savic, D.: Lost in optimization of water distribution systems? A literature review of system operations. Environ. Model. Softw. 93, 209–254 (2017)CrossRefGoogle Scholar
  12. 12.
    Pasha, M.F.K., Lansey, K.: Optimal pump scheduling by linear programming. In: Proceedings of World Environmental and Water Resources Congress 2009 - World Environmental and Water Resources Congress 2009: Great Rivers, vol. 342, pp. 395–404 (2009)Google Scholar
  13. 13.
    Powell, W.B.: Approximate Dynamic Programming: Solving the Curses of Dimensionality. Wiley, New York (2007)Google Scholar
  14. 14.
    Puterman, M.: Markov Decision Processes. Discrete Stochastic Dynamic Programming. Wiley, New York (1994)zbMATHGoogle Scholar
  15. 15.
    Shabani, S., Candelieri, A., Archetti, F., Naser, G.: Gene expression programming coupled with unsupervised learning: a two-stage learning process in multi-scale, short-term water demand forecasts. Water 10(2), 142 (2018)CrossRefGoogle Scholar
  16. 16.
    Sterling, M.J.H., Coulbeck, B.: A dynamic programming solution to the optimization of pumping costs, in Hybrid genetic algorithm in the optimization of energy costs in water supply networks. ICE Proc. 59(2), 813–818 (1975)Google Scholar
  17. 17.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An introduction – Adaptive Computation and Machine Learning. MIT Press, Cambridge (1998)Google Scholar
  18. 18.
    Wiering, M., Van Otterlo, M.: Reinforcement Learning - State-of-the- Art, 1st edn. Springer, Berlin (2012)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Antonio Candelieri
    • 1
    Email author
  • Riccardo Perego
    • 1
  • Francesco Archetti
    • 1
    • 2
  1. 1.Department of Computer Science, Systems and CommunicationUniversity of Milano-BicoccaMilanItaly
  2. 2.Consorzio Milano RicercheMilanItaly

Personalised recommendations