A unified observer with stochastic and deterministic robustness is developed in this paper so that an observer is less sensitive to both stochastic and deterministic uncertainties. For stochastic robustness, the norm of the observer gain and the lower bound of the observer decay rate are shown to be design factors which can minimize the upper bound of the estimation error variance. For deterministic robustness, the L-2 norm-based condition number of the observer eigenvector matrix is utilized to address robust estimation performance against deterministic uncertainties. In order to justify the proposed method, a graphical approach is first introduced, and then a multi-objective optimization problem including linear matrix inequality constraints is formulated to provide the unified robustness.