Aarhus University Seal / Aarhus Universitets segl

Erik Jørgensen

Markov Limid processes for representing and solving renewal problems

Research output: Contribution to journal/Conference contribution in journal/Contribution to newspaperJournal articleResearchpeer-review

  • Erik Jørgensen
  • Anders R. Kristensen, University of Copenhagen, Denmark
  • Dennis Nilsson, University of Copenhagen, Denmark
In this paper a new tool for simultaneous optimisation of decisions on multiple time scales is presented. The tool combines the dynamic properties of Markov decision processes with the flexible and compact state space representation of LImited Memory Influence Diagrams (Limids). A temporal version of Limids, TemLimids, is defined by adding time-related functions to utility nodes. As a result, expected discounted utility, as well as expected relative utility might be used as optimisation criteria in TemLimids. Optimisation proceeds as in ordinary Limids. A sequence of such TemLimids can be used to model a Markov Limid Process, where each TemLimid represents a macro action. Algorithms are presented to find optimal plans for a sequence of such macro actions. Use of algorithms is illustrated based on an extended version of an example from pig production originally used to introduce the Limid concept
Original languageEnglish
JournalAnnals of Operations Research
Pages (from-to)63-84
Publication statusPublished - Aug 2014

    Research areas

  • Markov decision process, MDP, Multilevel hierarchical Markov process, MLHMP, Limited Memory Influence Diagram, LIMID, Pig production

See relations at Aarhus University Citationformats

ID: 50962839