This paper analyzes the convergence of optimal control problems for an evolution equation in a finite time-horizon [0, T] toward the limit steady state ones as T -> infinity. We focus on linear problems. We first consider linear time-independent finite-dimensional systems and show that the optimal controls and states exponentially converge in the transient time (as T tends to infinity) to the ones of the corresponding steady state model. For this to occur suitable observability assumptions need to be imposed. We then extend the results to infinite-dimensional systems including the linear heat and wave equations with distributed controls.