As a quantitative criterion for privacy of "mechanisms" in the form of data-generating processes, the concept of differential privacy was first proposed in computer science and has later been applied to linear dynamical systems. However, differential privacy has not been studied in depth together with other properties of dynamical systems, and it has not been fully utilized for controller design. In this article, first we clarify that a classical concept in systems and control, input observability (sometimes referred to as left invertibility) has a strong connection with differential privacy. In particular, we show that the Gaussian mechanism can be made highly differentially private by adding small noise, if the corresponding system is less input observable. Next, enabled by our new insight into privacy, we develop a method to design dynamic controllers for the classic tracking control problem while addressing privacy concerns. We call the obtained controller through our design method the privacy-preserving controller. The usage of such controllers is further illustrated by an example of tracking the prescribed power supply in a dc microgrid installed with smart meters while keeping the electricity consumers' tracking errors private.