Soft sensors can predict values of a process variable y that is difficult to measure in real time. Adaptive mechanisms are applied to soft sensors to maintain their predictive ability. However, traditional adaptive soft sensors need a significant number of new y measurements. It is difficult to maintain the accuracy if the measurement interval is large. We propose two soft sensor models that produce accurate results with a small number of y measurements. We combined either a moving window technique or a just-in-time technique with a time difference model to handle changes of the slope between input variables X and y and shifts in X and y values. We analyzed a numerical simulation data set and a real industrial data set, demonstrating the superiority of the time difference model combined with a moving window technique.