A continuum model is proposed to study the morphological stability of epitaxial layers grown on compliant substrates lying on a viscous liquid. It is shown that the thickness of the substrate can significantly affect the stability of the film. When the film is very thin, long-range forces come into play, and they can stabilize or destabilize the film. In some special cases, long-range forces can compete with surface energy to form a critical thickness below which the film remains flat. This is consistent with the previous study. When the film is relatively thick in comparison with the substrate, efficient elastic relaxation may results in the existence of another critical film thickness above which the film is stable. (C) 2003 Elsevier B.V. All rights reserved.