Posttranslational modifications of the microtubule cytoskeleton have emerged as key regulators of cellular functions, and their perturbations have been linked to a growing number of human pathologies. Tubulin glycylation modifies microtubules specifically in cilia and flagella, but its functional and mechanistic roles remain unclear. In this study, we generated a mouse model entirely lacking tubulin glycylation. Male mice were subfertile owing to aberrant beat patterns of their sperm flagella, which impeded the straight swimming of sperm cells. Using cryo-electron tomography, we showed that lack of glycylation caused abnormal conformations of the dynein arms within sperm axonemes, providing the structural basis for the observed dysfunction. Our findings reveal the importance of microtubule glycylation for controlled flagellar beating, directional sperm swimming, and male fertility.