As a donor and acceptor separately, Nb5+ and Al3+ are used to substitute Ti4+ in the ixiolite ZnTiNb2O8 system for the first time. The dielectric responses in the terahertz range of this system are initially studied based on the data from terahertz time-domain transmitted spectroscopy. Combined with ligand field theory, the formation of a secondary phase of ZnAl2O4 is reasonably explicated. Then, the origins of the lower dielectric loss and the terahertz wave absorption coefficient are determined to be the high chemical bond covalency and effective phase control. For the composition of ZnTi0.85(Al0.5Nb0.5)(0.15)Nb2O8 sintered at 1180 degrees C, a low dielectric loss (THz) and absorption coefficient (THz) are obtained, which make this kind of material propitious for developing terahertz dielectric devices.