Most classical fault diagnosis methods such as principal component analysis (PCA), are extracting comprehensive information to represent data features in fault diagnosis. In comparison, Non-Negative Matrix Factorization (NMF) is a method for dimension reduction and feature extraction, and because its characteristic matrix has sparsity, this method is superior in the ability of extracting the local feature and suppressing noises; however, the NMF method is not applicable for dynamic industrial processes. In this paper, we introduce the past information of industrial processes for fault diagnosis, proposing Canonical Variate Analysis - Non-Negative Matrix Factorization (CVA-NMF) methods to improve the dynamic performance of NMF. The experimental results via TE process indicate that the proposed approach could handle a dynamic production process such as Fault 2 and Fault 5 and retain the superior performance of NMF.