Thermal expansion coefficients of perovskite oxides LaMO3 and La0.9Sr0.1MO3 with different size of B-site ions were measured, and the doping effect to the phase transition from orthorhombic to rhombohedral structure was discussed in terms of the size of the B-site ions and the tolerance factor of the perovskites. The transition temperature from orthorhombic to rhombohedral structure of LaGaO3 rose with Mg doping, but it did not change significantly in the case of Sr doping. The thermal expansion coefficient of LaScO3, and LaInO3 and LaAlO3 showed no thermal expansion anomaly from orthorhombic to rhombohedral structure, The transition temperature from orthorhombic to rhombohedral structure for LaMO3 and La0.9Sr0.1MO3 with different size of B-site ions linearly decreased with the tolerance factor, indicating that the perovskite with a small tolerance factor is distorted, resulting in the higher transition temperature. The perovskite with a small tolerance factor was shown to be distorted using the pair correlation function of oxygen-oxygen obtained by molecular dynamics calculation.