SIAM Journal on Control and Optimization, Vol.53, No.6, 3503-3533, 2015
RANDOMIZED AND RELAXED STRATEGIES IN CONTINUOUS-TIME MARKOV DECISION PROCESSES
One of the goals of this article is to describe a wide class of control strategies, which includes the traditional relaxed strategies, as well as the so called randomized strategies which appeared earlier only in the framework of semi-Markov decision processes. If the objective is the total expected cost up to the accumulation of jumps, then without loss of generality one can consider only Markov relaxed strategies. Under a simple condition, the Markov randomized strategies are also sufficient. An example shows that the mentioned condition is important. Finally, without any conditions, the class of so called Poisson-related strategies is also sufficient in the optimization problems. All the results are applicable to the discounted model, they may be useful also for the case of long-run average cost.
Keywords:continuous-time Markov decision process;total cost;discounted cost;occupation measure;relaxed strategy;randomized strategy