A way to prepare Poly (phenylene sulfide) (PPS) nanocomposite was introduced in the paper. The nanocomposite of PPS/CaCO3 can be prepared by melt mixing process. The dispersion of the CaCO3 nanoparticles in PPS was good when filler content below 5 wt%. Differential scanning calorimeter (DSC) and small-angle light scattering (SALS) results indicated that the CaCO3 nanoparticles could induce the nucleation but retard the mobility of polymer chains. The results of mechanical tests showed that a small amount of nanoparticles has resulted in a slight improvement in the tensile strength and a significantly 300% increase in the fracture toughness. We believe that the CaCO3 nanoparticles can act as stress concentration sites, which can promote cavitation at the particles boundaries during loading. The cavitation can trigger mass plastic deformation of the matrix, leading to much improve fracture toughness.