This paper investigated a novel mean grain size characterization method via laser ultrasonic. The IF steel samples with different grain sizes were conducted with different heat treatments, respectively, and each sample was observed by EBSD and tested by laser ultrasonic. The laser ultrasonic signal was decomposed by wavelet packet transform and sorted using correlation analysis. Then the optimal components were selected to reconstruct the new signal and obtain the energy attenuation coefficient. Finally, the novel mean grain size prediction model was established according to the Rayleigh scattering and absorption attenuation. These results show that the maximum prediction error could be reduced to 5.74% by our new method, which is more precise than the traditionally used methods. Our new method improves the mean grain size evaluation to a more precise level. In addition, compared with the conventional mean grain size acquisition methods, the laser ultrasonic method has the advantages of non-contact and high efficiency, and it is a nondestructive testing method which could be used in online testing.