SIAM Journal on Control and Optimization, Vol.49, No.5, 1938-1960, 2011
RECURRENCE AND TRANSIENCE OF OPTIMAL FEEDBACK PROCESSES ASSOCIATED WITH BELLMAN EQUATIONS OF ERGODIC TYPE
The paper is concerned with ergodic-type Bellman equations arising typically in linear (exponential) quadratic Gaussian control. We are interested in giving recurrence-transience criteria for associated optimal feedback diffusions in terms of qualitative properties of solutions to the Bellman equation. To establish such criteria, we propose a new approach which is based on the Lyapunov method. It turns out that a certain convexity of the equation plays a key role in our arguments.
Keywords:criteria for transience or recurrence;ergodic-type Bellman equation;optimal feedback control;risk-sensitive control;Lyapunov method