Several recent studies link parental environments to phenotypes in subsequent generations. In this work, we investigate the mechanism by which paternal diet affects offspring metabolism. Protein restriction in mice affects small RNA (sRNA) levels in mature sperm, with decreased let-7 levels and increased amounts of 5' fragments of glycine transfer RNAs (tRNAs). In testicular sperm, tRNA fragments are scarce but increase in abundance as sperm mature in the epididymis. Epididymosomes (vesicles that fuse with sperm during epididymal transit) carry RNA payloads matching those of mature sperm and can deliver RNAs to immature sperm in vitro. Functionally, tRNA-glycine-GCC fragments repress genes associated with the endogenous retroelement MERVL, in both embryonic stem cells and embryos. Our results shed light on sRNA biogenesis and its dietary regulation during posttesticular sperm maturation, and they also link tRNA fragments to regulation of endogenous retroelements active in the preimplantation embryo.