Two stochastic adaptive control schemes, the stochastic gradient and modified least squares, are studied. We consider these for scalar ARMAX systems with general input delays. First, when the algorithms are based on generalized minimum variance control with reference tracking, sufficient conditions for stability and optimality are found. This is done using martingale convergence analysis. Secondly, we examine parameter convergence for each of the algorithms, and establish conditions for convergence of the parameter estimates to a random multiple of the true parameters.