Chemical vapor deposition is widely used in integrated circuit fabrication. With the increasingly stringent requirements for product quality and yield, it is necessary to apply process monitoring, fault detection and eventually closed-loop control to the deposition process. The implementation of these tools requires an in situ sensor that can measure film properties in real time. This paper discusses an extended Kalman filter (EKF) based in situ sensor and its application in chemical vapor deposition of yttria-stabilized zirconia (YSZ) on a silicon substrate. The sensor measures reflectance from the film surface at 950 and 470 nm. The measurements were processed by the EKF to estimate film thickness, growth rate and surface roughness. The challenge with estimation of film microstructure like roughness lies in the process dynamic model for microstructure evolution at small scales, which in general is high-dimensional and nonlinear. In this work a simple but adaptive process dynamic model and a detailed optical sensor model were used to study how a simple process model could aid in estimation. Based on these models the observability of the process was discussed. The estimated film properties were verified with ex situ characterization using a scanning electron microscope (SEM) and an atomic force microscope (AFM). Finally sensitivity and robustness of EKF were discussed, along with our perspectives on future work on in situ sensing for thin film deposition. (c) 2006 Elsevier Ltd. All rights reserved.