Lithium-ion batteries have been widely used in many fields. The state-of-health is necessary and important for battery performance evaluation and lifetime prediction. A reliable state-of-health estimation is essential to help batteries work in a safe and suitable condition. In this paper, a novel state-of-health estimation approach is proposed for lithium-ion batteries based on statistical knowledge. An improved battery model, which combines the open-circuit-voltage modeling and the Thevenin equivalent circuit model, is proposed to improve the model accuracy and study the relation between internal parameters and states of the battery. The joint extended Kalman filter-recursive-least squares algorithm is employed to estimate battery state-of-charge and identify the model parameters and open-circuit-voltage simultaneously. Then a particle swarm optimization-least square support vector regression approach is employed to give a reliable state-of-health estimation result with high accuracy and good generalization ability, where the particle swarm optimization algorithm is used to improve the algorithm ability of global optimization. In order to verify the accuracy of the proposed method, static and dynamic current profile tests are carried out on lithium iron phosphate batteries in different aging levels. The experimental results indicate that the proposed method can present suitability for state-of-health estimation with high accuracy.