An optimal control problem with quadratic cost functional for a linear stochastic system with pointwise and distributed time delays in the state variable is considered. The case where the cost functional does not contain a control cost is treated. The latter means that the problem under consideration is a singular optimal control problem. This control problem is associated with a new optimal control problem for the same equation of dynamics. The cost functional in this new problem is the sum of the original cost functional and an integral of the square of the control with a small positive weighting coefficient. Thus, the new problem is a regular optimal control problem. Moreover, it is a cheap control problem. By using the control optimality conditions, the solution of this problem is reduced to solution of the set of four equations: one Riccati-type matrix ordinary differential equation, two Riccati-type matrix partial differential equations of the first order, and one trivial scalar ordinary differential equation. This set of equations is with deviating arguments, and it is singularly perturbed. An asymptotic solution of this set of equations is constructed and justified. Based on this asymptotic solution, an asymptotically suboptimal state-feedback control is constructed for the cheap control problem. Finally, it is shown that this control constitutes a minimizing sequence for the original problem. The infimum of the cost functional in the original optimal control problem also is obtained. An illustrative example is presented.