This paper presents a certainty equivalence design method with application for continuous-time stochastic systems with unknown parameters. The first aim is to approximate the observed process x = (x(t))(t >= 0), satisfying a stochastic differential equation with control, by choosing a control process (u(t))(t >= 0) to a stable Ornstein-Uhlenbeck process (x(t)(0))(t >= 0), driven by the same Wiener process. Based on observation of x for a given epsilon > 0 an adaptive control process u(epsilon) = (u(t)(epsilon))(t >= 0) is constructed such that the corresponding observed process x(u(epsilon)) = (x(t)(epsilon))(t >= 0) satisfies the relation sup(t >= t epsilon) E(x(t)(epsilon) - x(t)(0))(2) <= epsilon. The time t(epsilon) (epsilon is a threshold quantity) is found. A similar problem is solved for a stochastic delay differential equation with an unknown parameter. It is shown that the time t(epsilon) has equal rates of increase epsilon(-1) ln epsilon(-1) in both problems. Moreover, a target inequality, which ensures the stability of the controlled processes in the sense of the L-2-norm, is established. Similar problems for more general controlled multidimensional systems can be solved using sequential estimators of unknown parameters.