The separation of particles in centrifugal compounding force field is complex due to the interaction of centrifugal force, viscous drag force, inter-particle contact force and particle-particle collision force. In this work, the CFD-DEM modeling is applied for the first time in the study of particle separation in an innovative centrifugal separator. Some key features of the flow in centrifugal separator are described by using CFD modeling. The DEM is used to model the motion of discrete particles by applying Newton's laws of motion. The particle-fluid and particle-particle interaction forces are examined to explain the separation of particles by density. This approach provides a convenient way to better understand the flow and separation performance of particles in centrifugal compounding force field. Then, a central composite experimental design is used to investigate the relationship between the independent (rotation speed and fluidization water flow rate) variables and the responses (rho(50) and Ep), possible interactions between the independent variables and their effects on the separation performance of the rotation bowl. The results suggest that the fluidization water flow rate have more impact on rho(50) and Ep than rotation speed. The fluidization water flow rate improves the separation efficiency. (C) 2018 Elsevier B.V. All rights reserved.