Unsaturated polyester (UP)-toughened epoxy nanocomposites were prepared, and their effective mechanical and thermal properties were studied. Two types of organo-modified montmorillonite (OMMT) clays were used to prepare the nanocomposites. X-ray diffraction (XRD) and transmission electron microscopy (TEM) analysis showed the formation of exfoliated silicate layers in the UP-toughened epoxymatrix. Mechanical tests revealed that nanocomposites (containing 1 wt% OMMT clay) showed an increase in tensile strength to 13.8%, flexural strength to 10%, and impact strength to 4% compared with an UP-toughened epoxy blend. The effect of different heating rates on the curing behavior of UP-toughened epoxy nanocomposites was investigated using non-isothermal differential scanning calorimetry. The data were interpreted using the Kissinger and Flynn-Wall-Ozawa models to find the curing reaction parameter. The water uptake behavior for nanocomposites increased with the addition of OMMTs. Scanning electron microscopy micrographs indicated morphological changes in the impact fractured samples of UP-toughened epoxy nanocomposites.