This paper discusses an infinite-horizon linear quadratic (LQ) optimal control problem involving state-and control-dependent noise in singularly perturbed stochastic systems. First, an asymptotic structure along with a stabilizing solution for the stochastic algebraic Riccati equation (ARE) are newly established. It is shown that the dominant part of this solution can be obtained by solving a parameter-independent system of coupled Riccati-type equations. Moreover, sufficient conditions for the existence of the stabilizing solution to the problem are given. A new sequential numerical algorithm for solving the reduced-order AREs is also described. Based on the asymptotic behavior of the ARE, a class of O(root epsilon) approximate controller that stabilizes the system is obtained. Unlike the existing results in singularly perturbed deterministic systems, it is noteworthy that the resulting controller achieves an O(epsilon) approximation to the optimal cost of the original LQ optimal control problem. As a result, the proposed control methodology can be applied to practical applications even if the value of the small parameter epsilon is not precisely known.