To improve the tribological properties of benzoxazine (BOZ) resin, bismaleimides (BMI) resin is chosen as organic phase, hyperbranched polysilane functionalized SiO2 nanoparticles (HBPSi-SiO2) are chosen as inorganic modifiers to prepare HBPSi-SiO2/BOZ-BMI composites using high shear and ultrasonic processes. The effect of content of HBPSi-SiO2 on the mechanical properties and tribological properties of the composites are investigated. The results show that suitable addition of HBPSi-SiO2 can largely enhance the impact strength, reduce the friction coefficient, and wear rate of BOZ-BMI resin. Scanning electron microscopy is employed to research the wearing mechanism of materials. The severe wear of the BOZ pure resin is owing to fatigue wear, and the moderate wear of BOZ-BMI resin is attributed to adhesive wear. While, the mild wear of the composites with HBPSi-SiO2 is due to abrasive wear. (c) 2013 Wiley Periodicals, Inc. J. Appl. Polym. Sci., 2013