Using the innovation analysis method in the time domain, based on the autoregressive moving average (ARMA) innovation model, this paper presents a unified white noise estimation theory that includes both input and measurement white noise estimators, and presents a new steady-state optimal state estimation theory. Non-recursive optimal state estimators are given, whose recursive version gives a steady-state Kalman filter, where a new algorithm of the Kalman filter gain is proposed. Two new algorithms of the Kalman predictor gain are also derived. Local asymptotic stability of the Kalman filter is proved. The classical Kalman filtering theory is extended and modified. The method used covers unstable systems with correlation noises and singular transition matrix, and also covers the self-tuning white noise, signal and state estimators, when the noise statistics is unknown. To illustrate, three simulation examples are given.