IEEE Transactions on Automatic Control, Vol.46, No.12, 1984-1989, 2001
A note on optimality conditions for continuous-time Markov decision processes with average cost criterion
This note deals with continuous-time Markov decision processes with a denumerable state space and the average cost criterion. The transition rates are allowed to be unbounded, and the action set is a Borel space. We give a new set of conditions under which the existence of optimal stationary policies is ensured by using the optimality inequality. Our results are illustrated with a controlled queueing model. Moreover, we use an example to show that our conditions do not imply the existence of a solution to the optimality equations in the previous literature.
Keywords:average cost criterion;continuous-time Markov decision processes (MDPs);optimal stationary policies;optimality inequality