Adaptive Control of Practical Heat Pump Systems for Power System Flexibility Based on Reinforcement Learning

Ruihao Song, Daniel Zinsmeister, Thomas Hamacher, Haoran Zhao, Vladmir Terzija, Vedran Peric

Research output: Contribution to book/anthology/report/proceedingArticle in proceedingsResearchpeer-review


Modern power systems are under flexibility shortage because of high renewable penetration. As heating systems are increasingly integrated with electric power systems, heat pumps have become a valuable source of power system flexibility. However, utilizing the flexibility of heat pumps necessitates additional regulation system on the heat pump, which complicates their design. Many commercially available heat pump systems modulate through a relatively slow ramping process and suffer from significant input transport delays. Due to complex dynamical process in heat pumps, a traditional model-free closed-loop power controller, such as the proportional-integral-derivative type, may result in poor transient performance. In contrast, an open-loop control may provide faster transient response at the expense of significant steady-state error. The steady state error is especially problematic due to high non-linearity of heat pump power consumption with respect to working condition variables, such as source and sink media temperatures and mass flow levels. This paper proposes an reinforcement learning based open-loop control system that provides fast transient response but is adaptive in nature to compensate for the non-linearities arisen from changing working conditions. The impact of working condition changes is captured with the trained deep neural network that modifies the modulation input to minimize potential steady-state power tracking error.

Original languageEnglish
Title of host publication2023 International Conference on Power System Technology (PowerCon)
Publication date2023
ISBN (Electronic)979-8-3503-0022-2, 979-8-3503-0023-9
Publication statusPublished - 2023
Externally publishedYes
Event2023 International Conference on Power System Technology, PowerCon 2023 - Jinan, China
Duration: 21 Sept 202322 Sept 2023


Conference2023 International Conference on Power System Technology, PowerCon 2023
SponsorChinese Society for Electrical Engineering, IEEE Power and Energy Society, State Grid Corporation of China


  • adaptive control
  • heat pump
  • power system flexibility
  • reinforcement learning


Dive into the research topics of 'Adaptive Control of Practical Heat Pump Systems for Power System Flexibility Based on Reinforcement Learning'. Together they form a unique fingerprint.

Cite this