By converting waste heat into electricity, thermoelectric generators could be an important part of the solution to today's energy challenges. The compound Zn4Sb3 is one of the most efficient thermoelectric materials known. Its high efficiency results from an extraordinarily low thermal conductivity in conjunction with the electronic structure of a heavily doped semiconductor. Previous structural studies have been unable to explain this unusual combination of properties. Here, we show through a comprehensive structural analysis using single-crystal X-ray and powder-synchrotron-radiation diffraction methods, that both the electronic and thermal properties of Zn4Sb3 can be understood in terms of unique structural features that have been previously overlooked. The identification of Sb3- ions and Sb-2(4-) dimers reveals that Zn4Sb3 is a valence semiconductor with the ideal stoichiometry Zn13Sb10. In addition, the structure contains significant disorder, with zinc atoms distributed over multiple positions. The discovery of glass-like interstitial sites uncovers a highly effective mechanism for reducing thermal conductivity. Thus Zn4Sb3 is in many ways an ideal 'phonon glass, electron crystal' thermoelectric material.