A set-valued observer (also called guaranteed state estimator) produces a set of possible states based on output measurements and models of exogenous signals. In this paper, we consider the guaranteed state estimation problem for linear time-varying systems with a priori magnitude bounds on exogenous signals. We provide an algorithm to propagate the set of possible states based on output measurements and show that the centers of these sets provide optimal estimates in an l(infinity)-induced norm sense, We then consider the utility of set-valued observers for disturbance rejection with output feedback and derive the following general separation structure. An optimal controller can consist of a set-valued observer followed by a static nonlinear function on the observed set of possible states. A general construction of this function is provided in the scalar control case. Furthermore, in the special case of full-control, i.e., the number of control inputs equals the number of states, optimal output feedback controllers can take the form of an optimal estimate of the full-state feedback controller.