Deriving Explicit Control Policies for Markov Decision Processes Using Symbolic Regression

A. Hristov, J. W. Bosman, S. Bhulai, R. D. Van Der Mei

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

In this paper, we introduce a novel approach to optimizing the control of systems that can be modeled as Markov decision processes (MDPs) with a threshold-based optimal policy. Our method is based on a specific type of genetic program known as symbolic regression (SR). We present how the performance of this program can be greatly improved by taking into account the corresponding MDP framework in which we apply it. The proposed method has two main advantages: (1) it results in near-optimal decision policies, and (2) in contrast to other algorithms, it generates closed-form approximations. Obtaining an explicit expression for the decision policy gives the opportunity to conduct sensitivity analysis, and allows instant calculation of a new threshold function for any change in the parameters. We emphasize that the introduced technique is highly general and applicable to MDPs that have a threshold-based policy. Extensive experimentation demonstrates the usefulness of the method.

Original languageEnglish
Title of host publicationVALUETOOLS '20
Subtitle of host publicationProceedings of the 13th EAI International Conference on Performance Evaluation Methodologies and Tools
PublisherAssociation for Computing Machinery
Pages41-47
Number of pages7
ISBN (Electronic)9781450376464
DOIs
Publication statusPublished - May 2020
Event13th EAI International Conference on Performance Evaluation Methodologies and Tools, VALUETOOLS 2020 - Tsukuba, Japan
Duration: 18 May 202020 May 2020

Publication series

NameACM International Conference Proceeding Series

Conference

Conference13th EAI International Conference on Performance Evaluation Methodologies and Tools, VALUETOOLS 2020
Country/TerritoryJapan
CityTsukuba
Period18/05/2020/05/20

Keywords

  • Closed-form approximation
  • Genetic program
  • Markov Decision Processes
  • Optimal control
  • Symbolic regression
  • Threshold-Type policy

Fingerprint

Dive into the research topics of 'Deriving Explicit Control Policies for Markov Decision Processes Using Symbolic Regression'. Together they form a unique fingerprint.

Cite this