Boehmite nanoparticles show great potential in improving the matrix dominated mechanical properties of fiber reinforced polymers. For the material design process and the prediction of the nanocomposite's properties, knowledge about the material behavior of the constituent phases and their interactions is crucial. Since the chemical composition of the particle surface can strongly affect the interaction between particle and matrix, the influence of particle surface modification and mass fraction on mechanical properties (tensile modulus, tensile strength, fracture toughness) and failure mechanisms of nanoparticle reinforced epoxy resins is investigated using experimental and numerical methods. In the present work, unmodified and thus chemically reactive boehmite particles are compared to acetic acid modified particles with supposedly lower chemical reactivity and thus worse particle-matrix bonding. A linear relationship between particle mass fraction and tensile modulus/fracture toughness is experimentally found with a maximum increase of 26% in tensile modulus and 62% in critical energy release rate for a particle content of 15 wt%. Furthermore, the experiments indicate that the acetic acid surface modification increases the tensile modulus (up to 6% compared to the unmodified boehmite particles), but at the same time not significantly affects the tensile strength or the fracture toughness. Molecular Dynamic Finite Element Method (MDFEM) simulations are conducted to identify and understand the mechanisms induced by nanoparticles. The material properties of both, modified and unmodified, nanoparticle systems are discussed in relation to the change of particle-matrix bonding strength and interphase morphology. While simulation results of the unmodified system show an outstanding agreement with the experiments, the acetic acid modified system deviates significantly. In conclusion, it seems that additional effects have to be considered to completely understand the material behavior. (C) 2018 Elsevier Ltd. All rights reserved.