SIAM Journal on Control and Optimization, Vol.58, No.1, 192-214, 2020
ON REDUCING A CONSTRAINED GRADUAL-IMPULSIVE CONTROL PROBLEM FOR A JUMP MARKOV MODEL TO A MODEL WITH GRADUAL CONTROL ONLY
In this paper we consider a gradual-impulsive control problem for continuous-time Markov decision processes (CTMDPs) with total cost criteria and constraints. We develop a simple and useful method, which reduces the concerned problem to a standard CTMDP problem with gradual control only. This allows us to derive straightforwardly and under a minimal set of conditions the optimality results (sufficient classes of control policies, as well as the existence of stationary optimal policies) for the original constrained gradual-impulsive control problem.
Keywords:continuous-time Markov decision processes;impulse-gradual control;reduction method;problem with constraints