Composite cerium-doped anatase nanophases with an average diameter of 20 nm were prepared via sol gel method followed by surface modification with stearic acid, abbreviated SA/Ce-TiO2. The microstructure of materials was characterized by XRD, FTIR, and SEM. Furthermore, the antiwear and friction-reducing capacities of SA/Ce-TiO2 as lubrication additives in rapeseed oil were evaluated on a four-ball tribotester. The chemical characteristics, elemental composition, and morphology of worn surfaces of steel balls were investigated by XPS and SEM. The results show that SA/Ce-TiO2 could obviously improve the antiwear and friction-reducing capacities of rapeseed oil, especially for the nanophases doped with 2.0 and 3.0% cerium in molar ratio, respectively. The excellent tribological performance of SA/Ce-TiO2 in rapeseed oil can be attributed to the formation of a complex boundary lubrication film mainly consisting of Fe2O3, TiO2, and CeO2 as well as the formation of an adsorption film of SA and base oil on the friction surfaces.