Graphene-based nanomaterials show great potential as tougheners for thermosetting resins because they offer effective toughening of resins at an extremely low loading level with minimal change in T-g or modulus. However, commercial resin formulations sometimes contain inorganic fillers that could affect the toughening behavior of graphene-based additives. In this study, the effects of combining modified graphene oxide (mGO) with several widely used filler materials (fumed silica, kaolin day, and calcium carbonate) on toughening of vinyl ester resins was evaluated. The relative size of additional filler particles compared to mGO particles strongly influences the toughening effects. mGO can toughen the resin if the other filler particles are much smaller and well dispersed. In contrast, the addition of mGO to filler-containing vinyl ester resin may decrease the fracture toughness if it reduces the size of the filler inclusion. Therefore, it is important to know the behavior of a filler in a resin before applying graphene-based toughener to it.