In this article, we present a novel control approach for formally solving reach-avoid problems for constrained and disturbed nonlinear systems by optimizing over reachable sets. Reach-avoid problems arise in many modern control applications of safety-critical systems, such as autonomous driving and human-robot collaboration. They require us to control all states from a set of initial states in finite time into a final set while guaranteeing the satisfaction of state and input constraints despite the presence of disturbances and uncertain measurements. We optimize over reachable sets, thereby, simultaneously improving the control performance and guaranteeing constraint satisfaction for all solutions. Moreover, our new approach involves a novel combination of state-dependent feedforward and feedback control, which leads to better control performance compared to the existing approaches as demonstrated in several numerical examples. Our algorithm is particularly suited for computing motion primitives used in maneuver automata, which realizes fast and efficient online planning. The online applicability is supported by the simple structure of the resulting controller, which is a time-varying linear tracking controller.