Our PhD student Jose speaks at the Machine Learning for Science and Engineering Symposium in Pittsburgh (June 6-8, 2018) during the Civil Engineering track. He’s talk is entitled Deep Reinforcement Learning for Urban Energy Management and introduces his research in CityLearn:
Abstract: The building sector is one of the largest consumers of energy, accounting for about half of the total energy demand in many countries. As urban population increases, new concerns arise on how to make cities more sustainable, resilient, and livable. Integration of distributed renewable energy resources, combined with energy storage systems, can help buildings to consume energy more efficiently and reduce their dependency on the electrical grid. However, these energy systems must be controlled properly to ensure efficient operation. Demand response allows consumers to reduce their electrical consumption during periods of peak energy use. This reduces the peaks of electrical demand, and, consequently, the wholesale prices of electricity. Furthermore, the increasing amount of sensor data available from smart-meters and buildings is useful for control systems based on machine learning to improve the energy management in buildings using historical data. Deep reinforcement learning (RL) is a self-tuning control algorithm that can learn from real-time and historical data, and adapt to factors such as weather conditions, behavior of the occupants or thermal behavior of buildings. Its model-free nature means that no models need to be developed for individual buildings, as is the case for, e.g., model predictive controllers. This makes deep RL scalable and, thus, very attractive for urban scale implementation, particularly for demand response in residential buildings.
Therefore, in our research, we investigate deep RL to coordinate buildings as multi-agent systems for demand response. To achieve this, we have developed CityLearn, a simulation environment based on CitySim, a simulator for urban scale energy analysis, and TensorFlow, a popular machine learning library. TensorFlow provides the simulator with efficient implementations of advanced machine learning algorithms, and CitySim allows analyzing urban energy models. We have applied CityLearn in two case studies: First, for a single building, we minimized the energy consumed from the electrical grid by a heat pump, in combination with a chilled water storage tank, and a photovoltaic array. We demonstrated how deep RL can self-tune itself to adapt to changes on the demand-side (building refurbishment) and on the supply-side (adding PV panels). In the second case study, two buildings are competing against each other as a multi-agent system to reduce their shared cost of electricity, which increases if both building consume at the same time. Additionally, we are also further extending CityLearn by integrating data-driven models that predict how occupants modify the temperature setpoints in the buildings. The result will be a deep RL controller that adapts to human patterns of energy use to maximize comfort and reduces the cost of electricity. CityLearn will allow to study these topics on an urban scale with many individual human patterns as well as a variety of buildings competing or collaborating with each other for resources.