Spherical nanoparticles (NPs) uniformly grafted with polymer chains have recently been shown to assemble into anisotropic phases like strings and sheets. Here we investigated the underlying basis for anisotropic sow interactions between polymer-grafted NPs in a polymer matrix by computing via molecular dynamics simulations the potential of mean force (PMF), and its three-body contribution, for a test NP interacting with a NP-dimer along a set of reaction coordinates differing in their orientation with respect to the dimer axis. The polymer-mediated portions of the PMF and of the three-body contribution were both found to be highly repulsive and anisotropic with the degree of repulsion rising with increasing angular deviation from the dimer axis. The anisotropy was shown to arise from the expulsion of polymer grafts from in between the dimer NPs which leads to a gradient in the graft segmental density around the dimer from its contact point to its poles. This effect produces a concomitant gradient in steric repulsion between test and dimer NP grafts, a significant portion of which is however negated by an opposing gradient in depletion attraction between NPs due to the matrix. The anisotropy in the polymer-mediated PMF was observed to be particularly strong for NP polymer systems with long grafts, high grafting densities, and short matrix chains. The overall PMFs allowed us to compute the free energies of formation of two-and three-particle clusters, yielding a phase diagram in graft length grafting density parameter space analogous to that observed experimentally for the dispersed, stringlike, and sheetlike phases of NPs. The PMFs also revealed possible existence of a stable dimer phase that remains to be tested experimentally. Taken together, this study illustrates how the deformability of NP grafts can introduce novel anisotropic interactions between otherwise isotropic NPs with far-reaching consequences in NP assembly.