The fastest possible collective response of a quantum many-body system is related to its excitations at the highest possible energy. In condensed matter systems, the time scale for such "ultrafast" processes is typically set by the Fermi energy. Taking advantage of fast and precise control of interactions between ultracold atoms, we observed nonequilibrium dynamics of impurities coupled to an atomic Fermi sea. Our interferometric measurements track the nonperturbative quantum evolution of a fermionic many-body system, revealing in real time the formation dynamics of quasi-particles and the quantum interference between attractive and repulsive states throughout the full depth of the Fermi sea. Ultrafast time-domain methods applied to strongly interacting quantum gases enable the study of the dynamics of quantum matter under extreme nonequilibrium conditions.