We have published a Tutorial on Reinforcement Learning (RL) in the forthcoming Handbook of Sustainable and Resilient Infrastructure published by Routledge. We argue that model-free algorithms, such as RL, are particularly useful for controlling buildings systems (HVAC, lighting, etc), because they adapt themselves to the individual characteristics of a building, while converging to optimal or close-to-optimal behavior. The handbook is edited by Paolo Gardoni from the University of Illinois at Urbana-Champaign and aims to coalesce work from a large and diverse group of contributors across a wide range of disciplines including engineering, technology and informatics, urban planning, public policy, economics, and finance.
Our contribution is in Part IX: Smart Cities and the Role of Information and Communication Technologies
Nagy, Z., Park, J.Y., and Vazquez-Canteli, J.R. (2018). “Reinforcement learning for intelligent environments: A Tutorial,” In Paolo Gardoni (Ed.), Handbook of Sustainable and Resilient Infrastructure, Routledge.
There is no clear definition of what a smart building constitutes, i.e., what makes a building smart. In fact, since its inception in the early 1980, the term has had many different notions and meanings, largely depending on the area of study. In this chapter, we employ the approach that a smart or intelligent building, or, more generally, environment, is one that allows it to interact with its occupants. Based on this assumption we put forward that idea that one particular type of machine learning algorithms, namely reinforcement learning is particularly suitable for implementation and application in the built environment for two reasons: 1) it is a model-free approach requiring no prior knowledge of the system dynamics, and 2) it is inherently based on the notion of interaction, which happens to occur frequently, especially between occupants and building systems. Therefore, in this chapter, we provide an introduction to reinforcement learning in a tutorial form. We discuss the most important variations, the implications of particular parameter choices, and we provide typical algorithms. Finally, we discuss some challenges and opportunities of applying reinforcement learning in the built environment.