In this study, randomly oriented single-walled carbon nanotubes (SWCNTs)/epoxy nanocomposites were fabricated by tip sonication with the aid of a solvent and subsequent casting. Two different curing cycles were used to study the role of the stiffness of the epoxy matrix on the tensile and thermal behavior of the composites. The addition of a small amount of SWCNTs (0.25 wt.%) in rubbery, i.e., soft matrices, greatly increased Young's modulus and tensile strength of the nanocomposites. The results showed that the tensile properties of soft epoxy matrices are much more influenced by the addition of carbon nanotubes than stiffer ones. The significant improvement in tensile properties was attributed to the excellent mechanical properties and structure of SWCNTs, an adequate dispersion of SWCNTs by tip sonication, and a stronger SWCNT/matrix interfacial adhesion in softer epoxy matrices. A slight improvement in the thermal stability of the nanocomposites was also observed.