Based on the milling force testing system, we have experimented with the waved-edge milling insert and the great milling insert developed by Harbin University of Science and Technology, the American Metal milling insert, the ISCAR hackle milling insert and the flat milling insert, and then compare the insert milling capability of various grooves milling inserts. We have discussed the dynamic milling force under the incontinuous cutting condition, especially the characteristic of time and frequency. Based on the power spectrum analysis of the five types of milling inserts, we have found that the insert whose groove is formed from the curve or curly plane of the period function has an advantage on minishing the milling force and resisting to vibration. The right insert geometry parameters have great effect on improving the cutting capability. By the power spectrum analysis, we found that the complex 3-D grooves milling insert is superior to the flat milling insert on minishing milling force and vibration. The analysis result proves that using power spectrum analysis in groove optimization is viable.