Entropy-stabilized oxides (ESOs) provide an alternative route to novel materials discovery and synthesis. It is, however, a challenge to demonstrate that the constituent elements in an entropy-stabilized crystal are homogeneously and randomly dispersed among a particular sublattice, resulting in a true solid solution with no evidence of local order or clustering. In this work, we present the application and analysis of extended X-ray absorption fine structure (EXAFS) on the prototype ESO composition MgxNixCoxCuxZnxO (x=0.2). In so doing, we can quantify the local atomic structure on an element-by-element basis. We conclude that local bond lengths between metal and oxygen vary around each absorbing cation, with notable distortion around the Cu-O polyhedra. By the second near neighbor (i.e., the cation-cation pair), interatomic distances are uniform to the extent that the collected data can resolve. Crystal models that best fit the experimental scattering data include cations that are distributed randomly on an FCC sublattice with minimal positional disorder, with an interleaved FCC anion sublattice with oxygen ions displaced from the ideal locations to accommodate the distortions in the cation polyhedra. Density functional theory calculations of the ESO system yield a significant broadening in the positional distribution for the oxygen sublattice compared to that for the cation sublattice for all peaks, showing consistency with the conclusion from the experimental data that the distortion from an ideal rock salt structure occurs primarily through disorder in the oxygen sublattice.