An algorithm for tracking time-varying parameters is studied. It is shown to be exponentially stable independently of the choice of algorithm parameters. We derive some useful nonasymptotic estimates for the tracking error and develop the asymptotically optimal version of the method.