In an effort to assess the performance of newer estimation algorithms, many prior publications have presented comparative studies where the Extended Kalman Filter (EKF) failed. This is because the EKF's design parameters were sometimes chosen arbitrarily and with little consideration of their role and impact on filter performance. This paper demonstrates in a tutorial way that EKF failure can often be avoided by a systematic design of its parameters, i.e. its covariance matrices. Particular focus is on the systematic selection of proper initial state and process noise covariance matrices. As in practice, model parameters are first identified from plant measurements. The covariance of these model parameters is subsequently required to calculate the proposed time-varying process noise covariance matrix for the EKF. Finally, the proposed EKE design is evaluated on a popular reactor example process from the literature and converges in all simulation runs. From this extensive set of simulations and by comparison with several out-of-the-box versions of the constrained Unscented Kalman Filter (UKF), we conclude that there exists a systematic way to design the EKF for a very satisfactory performance. Therefore, the EKF's design parameters need not be tuned ad hoc by trial and error.