SIAM Journal on Control and Optimization, Vol.48, No.8, 5178-5192, 2010
ON THE CONVERGENCE, LOCK-IN PROBABILITY, AND SAMPLE COMPLEXITY OF STOCHASTIC APPROXIMATION
It is shown that under standard hypotheses, if stochastic approximation iterates remain tight, they converge with probability one to what their o.d.e. limit suggests. A simple test for tightness (and therefore a.s. convergence) is provided. Further, estimates on lock-in probability, i.e., the probability of convergence to a specific attractor of the o.d.e. limit given that the iterates visit its domain of attraction, and sample complexity, i.e., the number of steps needed to be within a prescribed neighborhood of the desired limit set with a prescribed probability, are also provided. The latter improve significantly upon existing results in that they require a much weaker condition on the martingale difference noise.
Keywords:stochastic approximation;tightness of iterates;almost sure convergence;lock-in probability;sample complexity