Download PDFOpen PDF in browser

Autonomous Control of Urban Storm Water Networks Using Reinforcement Learning

5 pagesPublished: September 20, 2018

Abstract

We investigate the real-time and autonomous operation of a 12 km2 urban storm water network, which has been retrofitted with sensors and control valves. Specifically, we evaluate reinforcement learning, a technique rooted in deep learning, as a system-level control methodology. The controller opens and closes valves in the system, which enhances the performance in the storm water network by coordinating the discharges amongst spatially distributed storm water assets (i.e. detention basins and wetlands). A reinforcement learning control algorithm is implemented to control the storm water network across an urban watershed. Results show that control of valves using reinforcement learning shows great potential, but extensive research still needs to be conducted to develop a fundamental understanding of control robustness. We specifically discuss the role and importance of the reward function (i.e. heuristic control objective), which guides the autonomous controller towards achieving the desired water shed scale response.

Keyphrases: real time control, reinforcement learning, stormwater systems

In: Goffredo La Loggia, Gabriele Freni, Valeria Puleo and Mauro De Marchis (editors). HIC 2018. 13th International Conference on Hydroinformatics, vol 3, pages 1465-1469.

BibTeX entry
@inproceedings{HIC2018:Autonomous_Control_Urban_Storm,
  author    = {Abhiram Mullapudi and Branko Kerkez},
  title     = {Autonomous Control of Urban Storm Water Networks Using Reinforcement Learning},
  booktitle = {HIC 2018. 13th International Conference on Hydroinformatics},
  editor    = {Goffredo La Loggia and Gabriele Freni and Valeria Puleo and Mauro De Marchis},
  series    = {EPiC Series in Engineering},
  volume    = {3},
  publisher = {EasyChair},
  bibsource = {EasyChair, https://easychair.org},
  issn      = {2516-2330},
  url       = {/publications/paper/M6sB},
  doi       = {10.29007/hx4d},
  pages     = {1465-1469},
  year      = {2018}}
Download PDFOpen PDF in browser