IEEE Transactions on Automatic Control, Vol.45, No.3, 413-420, 2000
Stability and instability of limit points for stochastic approximation algorithms
It Is shown that the limit points of a stochastic approximation (SA) algorithm compose a connected set. Conditions are given to guarantee the uniqueness of the limit point for a given initial value. Examples are provided wherein {x(n)} of SA algorithm converges to a limit (x) over bar independent of initial values, hut (x) over bar is unstable for the differential equation (x) over dot = f(x) with a nonnegative Lyapunov function. Finally, sufficient conditions are given for stability of (x) over dot = f(x) at (x) over bar if {x(n)} tends to (x) over bar for any initial values.