A mathematical model is proposed to describe the multiphase flow in a dense-medium cyclone (DMC). In this model, the volume of fluid multiphase model is first used to determine the shape and position of the air core, and then the mixture multiphase model is employed to describe the flow of the dense medium (comprising finely ground magnetite in water) and the air core, where the turbulence is described by the Reynolds stress model. The results of fluid flow are finally used in the simulation of coal particle flow described by the stochastic Lagrangian particle tracking model. The validity of the proposed approach is verified by the reasonably good agreement between the measured and predicted results under different conditions. The flow features in a DMC are then examined in terms of factors such as flow field, pressure drop, particle trajectories, and separation efficiency. The results are used to explain the key characteristics of flow in DMCs, such as the origin of a short-circuit flow, the flow pattern, and the motion of coal particles. Moreover, the so-called surging phenomenon is examined in relation to the instability of fluid flow. The model offers a convenient method to investigate the effects of variables related to geometrical and operational conditions on the performance of DMCs.