Linear and nonlinear stability of a rotating viscoelastic fluid saturated porous layer heated from below is studied analytically. The modified Darcy-Oldroyd model that includes the time derivative and Coriolis terms is employed as a momentum equation. The onset criterion for both stationary and oscillatory convection is derived as a function of Taylor number, Darcy-Prandtl number and viscoelastic parameters. There is a competition between the processes of rotation, viscous relaxation and thermal diffusion that causes the convection to set in through oscillatory mode rather than stationary. The rotation inhibits the onset of convection in both stationary and oscillatory modes. The stress relaxation parameter destabilizes the system towards the oscillatory mode, while the strain retardation parameter enhances the stability. The effect of Darcy-Prandtl number on the stability of the system is also investigated. The nonlinear theory is based on the truncated representation of Fourier series method. The effect of rotation. stress relaxation and strain-retardation time and also the Darcy-Prandtl number on the transient heat transfer is presented graphically. (C) 2010 Elsevier Ltd. All rights reserved.