Domestication of a transposon (a DNA sequence that can change its position in a genome) to give rise to the RAG1-RAG2 recombinase (RAG) and V(D)J recombination, which produces the diverse repertoire of antibodies and T cell receptors, was a pivotal event in the evolution of the adaptive immune system of jawed vertebrates. The evolutionary adaptations that transformed the ancestral RAG transposase into a RAG recombinase with appropriately regulated DNA cleavage and transposition activities are not understood. Here, beginning with cryo-electron microscopy structures of the amphioxus ProtoRAG transposase (an evolutionary relative of RAG), we identify amino acid residues and domains the acquisition or loss of which underpins the propensity of RAG for coupled cleavage, its preference for asymmetric DNA substrates and its inability to perform transposition in cells. In particular, we identify two adaptations specific to jawed-vertebrates-arginine 848 in RAG1 and an acidic region in RAG2-that together suppress RAG-mediated transposition more than 1,000-fold. Our findings reveal a two-tiered mechanism for the suppression of RAG-mediated transposition, illuminate the evolution of V(D)J recombination and provide insight into the principles that govern the molecular domestication of transposons.