Multi-fluid flows are frequently thought of as being less stable than single phase flows. Consideration of different non-Newtonian models can give rise to different types of hydrodynamic instability. Here we show that with careful choice of fluid rheologies and flow paradigm, one can achieve multi-layer flows that are linearly stable for Re = infinity. The basic methodology consists of two steps. First we eliminate interfacial instabilities by using a yield stress fluid in one fluid layer and ensuring that for the base flow configurations studied we maintain an unyielded plug region at the interface. Secondly we eliminate linear shear instabilities by ensuring a strong enough Couette component in the second fluid layer, imposed via the moving interface. We show that this technique can be applied to both shear-thinning and visco-elastic fluids. (C) 2010 Elsevier B.V. All rights reserved.