This paper aims to disclose the noticeable effect of local tangential force on vibration performance in fractional-slot concentrated-winding permanent magnet synchronous machines (PMSMs). The mechanism to induce vibrations by tangential force is revealed at first. It is found that tangential force is able to deflect the yoke through the tooth's lever arm effect. This phenomenon is detailedly explained through finite element simulation and analytical description of the microscopic deformation in this paper. With the aid of a 10-pole/12-slot PMSM, the quantitative contribution of tangential force to the vibration is also investigated through simulation, vibration synthesis, and experiments. Results show that due to the strong lever arm effect and not so small force amplitudes as commonly expected, the contribution of tangential force to final vibration is comparable to that of radial force. Besides that, the synthesized vibrations indicate the contribution of tangential force decreases with the increase of non-zero spatial order, in a way similar to radial force. In order to underpin the findings, more situations such as with different pole-slot combinations are also analyzed at last.