The decay of an excited atom undergoing spontaneous photon emission into the fluctuating quantum-electrodynamic vacuum is an emblematic example of the dynamics of an open quantum system. Recent experiments have demonstrated that the gapped photon dispersion in periodic structures, which prevents photons in certain frequency ranges from propagating, can give rise to unusual spontaneous-decay behaviour, including the formation of dissipative bound states(1-3). So far, these effects have been restricted to the optical domain. Here we demonstrate similar behaviour in a system of artificial emitters, realized using ultracold atoms in an optical lattice, which decay by emitting matter-wave, rather than optical, radiation into free space. By controlling vacuum coupling and the excitation energy, we directly observe exponential and partly reversible non-Markovian dynamics and detect a tunable bound state that contains evanescent matter waves. Our system provides a flexible platform for simulating open-system quantum electrodynamics and for studying dissipative many-body physics with ultracold atoms(4-6).