This paper considers the problem of stabilizing a single-input-single-output (SISO) linear time-invariant (LTI) plant with known time delay using a low-order controller, such as a Proportional (P), a Proportional-Integral (PI), or a proportional-integral-derivative (PE)) controller. For the SISO LTI system with time delay, the closed-loop characteristic function is a quasipolynomial that possesses the following features: all its infinite roots are located on the left of certain vertical line of the complex plane, and the number of its unstable roots is finite. Necessary and sufficient conditions for the stability of LTI systems with time delay are first presented by employing an extended Hermite-Biehler Theorem applicable to quasi-polynomials. Based on the conditions, analytical algorithms are then proposed to compute the stabilizing sets of P, PI and PID controllers. The resulting characterizations of the stabilizing sets for P, PI and PID controllers are analogous to the Youla parameterization of all stabilizing controllers for plants without time delay. Numerical examples are provided to illustrate the proposed algorithm.