One of the most direct outcomes one could have envisioned from the two-dimensional (fifth-order) nonresonant Raman spectroscopy of liquids would have been a verdict on usefulness of instantaneous normal modes (INMs) as a basis for describing ultrafast liquid dynamics. Seeing the echo predicted by standard INM theory would have been persuasive evidence that this dynamics could really be thought of in terms of independent harmonic intermolecular vibrations. However, molecular dynamics calculations on liquid Xe show that there is no echo, implying that dynamical anharmonicities can have qualitative consequences even on ultrafast time scales-a notion seemingly inimical to the entire INM concept. What we show in this paper is that the fifth-order Raman spectrum can be understood within the confines of INM ideas, and from a fully molecular perspective, simply by including the contributions of the pure dephasing undergone by each INM mode. We show, in particular, that this dephasing stems from the adiabatic variation of the INM frequencies and of the cubic anharmonicity along each mode, and that lack of an echo can be understood from the magnitudes of the instantaneous anharmonicities alone. The resulting detailed picture of fifth-order Raman spectroscopy allows us, at least for liquid Xe, to assign a definitive mechanism for the origin of the signal; the spectrum is largely a measure of the liquid's dynamical anharmonicities and not of any nonlinear coupling of the liquid dynamics to the polarizability.