Recurrent Soft Actor Critic Reinforcement Learning for Demand Response Problems

Ulrich Ludolfinger, Daniel Zinsmeister, Vedran S. Perić, Thomas Hamacher, Sascha Hauke, Maren Martens

Research output: Contribution to book/anthology/report/proceedingArticle in proceedingsResearchpeer-review


Demand response problems are typically solved with rule-based or model predictive control solutions. While rule-based controls do not take into account the future development of uncertain variables, model predictive control solutions often require very accurate predictions. To solve demand response problems, which incorporate the inaccuracies of predictions in their decision making, deep reinforcement learning methods have become popular. For their implementation, current literature defines the demand response problem as a fully observable Markov decision process. However, the assumption of full observability is usually not satisfied in reality. An alternative idea is to describe the problem as partially observable and to use recurrency in the policy function. In this paper, we adapt this idea and propose a novel deep reinforcement learning control solution for demand response problems, based on the soft actor critic framework. Controlling a heat pump with a mixture of discrete and continuous action capabilities, we show that significant performance improvement can be achieved by using recurrency in the policy compared to a non recurrent policy function.

Original languageEnglish
Title of host publication2023 IEEE Belgrade PowerTech
Publication date2023
ISBN (Electronic)978-1-6654-8777-1, 978-1-6654-8779-5, 978-1-6654-8778-8
Publication statusPublished - 2023
Externally publishedYes
Event2023 IEEE Belgrade PowerTech, PowerTech 2023 - Belgrade, Serbia
Duration: 25 Jun 202329 Jun 2023


Conference2023 IEEE Belgrade PowerTech, PowerTech 2023
SponsorDIG Silent, EMTP, et al, Opal-RT Technologies, RTDS Technologies, Saturn Electric


  • Demand Response
  • Home Energy Management
  • Machine Learning
  • Partially Observable Markov Decision Process
  • Recurrent Soft Actor Critic
  • Reinforcement Learning


Dive into the research topics of 'Recurrent Soft Actor Critic Reinforcement Learning for Demand Response Problems'. Together they form a unique fingerprint.

Cite this