Herschel-Bulkley materials can be set in motion when a sufficiently high shear stress or body force is applied to them. We investigate the behaviour of a layer of Herschel-Bulkley fluid when it is suddenly tilted and subject to gravitational forces. The material's dynamic response depends on the details of its constitutive equation. When its rheological behaviour is viscoelastoplastic with no thixotropic behaviour, the material is set in motion instantaneously along its entire base. When its rheological behaviour involves two yield stresses (static and dynamic yield stresses), the material must be destabilised before it starts to flow. This problem is thus similar to a Stefan problem, with an interface that separates the sheared and unsheared regions and moves from top to bottom. We estimate the time needed to set the layer in motion in both cases. We also compare the solution to the local balance equations with the solution to the depth-averaged mass and momentum equations and show that the latter does not provide consistent solutions for this flow geometry. (C) 2017 Elsevier B.V. All rights reserved.