An approximation approach for the deviation matrix of continuous-time Markov processes with application to Markov decision theory

B.F. Heidergott, A. Hordijk, N. Leder

Research output: Contribution to JournalArticleAcademicpeer-review

192 Downloads (Pure)

Abstract

We present an update formula that allows the expression of the deviation matrix of a continuous-time Markov process with denumerable state space having generator matrix Q* through a continuous-time Markov process with generator matrix Q. We show that under suitable stability conditions the algorithm converges at a geometric rate. By applying the concept to three different examples, namely, the M/M/1 queue with vacations, the M/G/1 queue, and a tandem network, we illustrate the broad applicability of our approach. For a problem in admission control, we apply our approximation algorithm toMarkov decision theory for computing the optimal control policy. Numerical examples are presented to highlight the efficiency of the proposed algorithm. © 2010 INFORMS.
Original languageEnglish
Pages (from-to)918-932
Number of pages13
JournalOperations Research
Volume58
DOIs
Publication statusPublished - 2010

Fingerprint Dive into the research topics of 'An approximation approach for the deviation matrix of continuous-time Markov processes with application to Markov decision theory'. Together they form a unique fingerprint.

Cite this