Perturbation analysis of inhomogeneuous finite Markov chains

B.F. Heidergott, H. Leahu, A. Loepker, G. Pflug

Research output: Contribution to JournalArticleAcademicpeer-review


In this paper we provide a perturbation analysis of finite time-inhomogeneous Markov processes. We derive closed-form representations for the derivative of the transition probability at time t, with t > 0. Elaborating on this result, we derive simple gradient estimators for transient performance characteristics either taken at some fixed point in time t, or for the integrated performance over a time interval [0, t]. Bounds for transient performance sensitivities are presented as well. Eventually, we identify a structural property of the derivative of the generator matrix of a Markov chain that leads to a significant simplification of the estimators.
Original languageEnglish
Pages (from-to)255-273
JournalAdvances in Applied Probability
Issue number1
Publication statusPublished - 2016


Dive into the research topics of 'Perturbation analysis of inhomogeneuous finite Markov chains'. Together they form a unique fingerprint.

Cite this