The dentitions of extant fishes and land vertebrates vary in both pattern and type of tooth replacement. It has been argued that the common ancestral condition likely resembles the nonmarginal, radially arranged tooth files of arthrodires, an early group of armoured fishes. We used synchrotron microtomography to describe the fossil dentitions of so-called acanthothoracids, the most phylogenetically basal jawed vertebrates with teeth, belonging to the genera Radotina, Kosoraspis, and Tlamaspis (from the Early Devonian of the Czech Republic). Their dentitions differ fundamentally from those of arthrodires; they are marginal, carried by a cheekbone or a series of short dermal bones along the jaw edges, and teeth are added lingually as is the case in many chondrichthyans (cartilaginous fishes) and osteichthyans (bony fishes and tetrapods). We propose these characteristics as ancestral for all jawed vertebrates.