In this study, the cause of the boundary layer separation, the determination of the separated region and the effect of the boundary layer separation on ejector performance were comprehensively investigated. The boundary layer separation inside the ejector was studied by using CFD method. A steam ejector refrigeration experimental system was set up to verify the numerical model. The degree of boundary layer separation was determined by back pressure and the ejector geometry parameters. The cause of ejector failure was analyzed. The influence of the boundary layer separation on ejector performance under different throat diameters and the nozzle exit positions (NXP) were investigated in detail. The results indicated that in certain size range of the throat diameter or the NXP, the ejector worked in critical mode and has better pumping performance when the operating conditions were constant. In the case of a small or large throat diameter or a large NXP, the boundary layer separation became more severe, the formation of the shock wave and the choke would be destroyed and the mixed fluid would not overcome the back pressure to discharge from the ejector. The efficiency of the ejector descended to zero, the ejector failed. (C) 2018 Elsevier Ltd. All rights reserved.