Aarhus University Seal

Massimo Fiorentini

Comparison of online and offline deep reinforcement learning with model predictive control for thermal energy management

Research output: Contribution to journal/Conference contribution in journal/Contribution to newspaperJournal articleResearchpeer-review

  • Silvio Brandi, Polytechnic University of Turin
  • ,
  • Massimo Fiorentini
  • Alfonso Capozzoli, Polytechnic University of Turin

This paper proposes a comparison between an online and offline Deep Reinforcement Learning (DRL) formulation with a Model Predictive Control (MPC) architecture for energy management of a cold-water buffer tank linking an office building and a chiller subject to time-varying energy prices, with the objective of minimizing operating costs. The intrinsic model-free approach of DRL is generally lost in common implementations for energy management, as they are usually pre-trained offline and require a surrogate model for this purpose. Simulation results showed that the online-trained DRL agent, while requiring an initial 4 weeks adjustment period achieving a relatively poor performance (160% higher cost), it converged to a control policy almost as effective as the model-based strategies (3.6% higher cost in the last month). This suggests that the DRL agent trained online may represent a promising solution to overcome the barrier represented by the modelling requirements of MPC and offline-trained DRL approaches.

Original languageEnglish
Article number104128
JournalAutomation in Construction
Publication statusPublished - Mar 2022
Externally publishedYes

Bibliographical note

Funding Information:
The work of Silvio Brandi was made in the context of a Ph.D. scholarship at Politecnico di Torino funded by Enerbrain s.r.l.

Publisher Copyright:
© 2022 Elsevier B.V.

    Research areas

  • Building energy consumption, Building energy management, Deep reinforcement learning, Energy savings, HVAC control, Model predictive control

See relations at Aarhus University Citationformats

ID: 295412329