- Previous Article
- Next Article
- Table of Contents
IEEE Transactions on Automatic Control, Vol.53, No.4, 1076-1082, 2008
Event-based optimization of Markov systems
Recent research indicates that Markov decision processes (MDPs) and perturbation analysis (PA) based optimization can be derived easily from two fundamental performance sensitivity formulas. With this sensitivity point of view, an event-based optimization approach, including event-based sensitivity analysis and event-based policy iteration, was proposed via an example by X. R. Can (Discrete Event Dyn. Syst.: Theory Appl., vol. 15, pp. 169-197, 2005). This approach utilizes the special feature of a system and illustrates how the potentials can be aggregated using the special feature. The approach applies to many practical problems that do not fit well the standard MDP formulation. This note provides a mathematical formulation and proves the main results for this approach.
Keywords:Markov decision processes (MDPs);performance potentials;perturbation analysis (PA);policy gradients;policy iteration