Ethylene/methacrylic acid (E/MAA) ionomers contain polyethylene crystallites, amorphous polymer segments, and ionic aggregates. While the property changes observed upon neutralization of the MAA units are often attributed to the formation of ionic aggregates, no quantitative description currently exists for how these three structural motifs-alone or in combination-control any of these material properties. In this paper, we define such relationships for perhaps the most basic mechanical property of interest: the small-strain modulus. At temperatures just below the melting point of the primary crystallites, the ionomers can be satisfactorily described as two-phase composites of crystallites and ionically cross-linked rubber; however, at room temperature, the modulus is far higher than such a description predicts. We trace this effect to a synergy between the ionic aggregates and secondary crystallites, which together form percolated rigid pathways through the amorphous phase at room temperature. When the secondary crystallites melt, these paths break down and the simple two-phase composite description is recovered.