Control of messenger RNA (mRNA) decay rate is intimately connected to translation elongation, but the spatial coordination of these events is poorly understood. The Ccr4-Not complex initiates mRNA decay through deadenylation and activation of decapping. We used a combination of cryo-electron microscopy, ribosome profiling, and mRNA stability assays to examine the recruitment of Ccr4-Not to the ribosome via specific interaction of the Not5 subunit with the ribosomal E-site in Saccharomyces cerevisiae. This interaction occurred when the ribosome lacked accommodated A-site transfer RNA, indicative of low codon optimality. Loss of the interaction resulted in the inability of the mRNA degradation machinery to sense codon optimality. Our findings elucidate a physical link between the Ccr4-Not complex and the ribosome and provide mechanistic insight into the coupling of decoding efficiency with mRNA stability.