This paper proposes a novel approach to detection and isolation of faulty sensors in multivariate dynamic systems. After formulating the problem of sensor fault detection and isolation in a dynamic system represented by a state space model, we develop the optimal design of a primary residual vector for fault detection and a set of structured residual vectors for fault isolation using an extended observability matrix and a lower triangular block Toeplitz matrix of the system. This work is, therefore, a vector extension to the earlier scalar-based approach to fault detection and isolation. Besides proposing a new algorithm for consistent identification of the Toeplitz matrix from noisy input and output observations without identifying the state space matrices {A, B, C, D}of the system, the main contributions of this newly proposed fault detection and isolation scheme are: (1) a set of structured residual vectors is employed for fault isolation; (2) after determination of the maximum number of multiple sensors that are most likely to fail simultaneously, a unified scheme for isolation of single and multiple faulty sensors is proposed; and (3) the optimality of the primary residual vector and the structured residual vectors is proven. We prove the advantage of our newly proposed vector-based scheme over the existing scalar element-based approach for fault isolation and illustrate its practicality by simulated and experimental evaluation on a multivariate pilot scale, computer interfaced system.