The reaction-induced phase separation in a tetrafunctional epoxy-cyclic anhydride system modified with polysulfone (PSF) was followed by optical microscopy (OM), light scattering (LS), and scanning electron microscopy (SEM). The selected system was N,N,N',N'-tetraglycidyl-4,4'-diamino diphenylmethane cured with methyl tetrahydrophthalic anhydride, in the presence of variable PSF concentrations. The different experimental techniques allow us to establish the phase separation mechanism. For modifier concentrations close to the critical point, 10 and 15 wt% PSF, phase separation proceeded by spinodal demixing (SD). For a modifier concentration much lower than the critical point, 5 wt% PSF, phase separation occurred via the nucleation and growth (NG) mode. For 7.5 wt% PSF, depending on the cure temperature, SD or NG was observed. Dynamic mechanical behavior of the resulting materials had been discussed based on fractionation of different species during the phase separation process. The fracture toughness increased significantly when bicontinuous (10 wt% PSF) or phase-inverted (15 wt% PSF) structures were generated. For mixtures containing 15 wt% PSF, the dependence of fracture toughness on the stoichiometric ratio (anhydride groups/epoxy groups) was analyzed. (C ) 1999 John Wiley & Sons, Inc.