The high carrier mobility of graphene(1-4) is key to its applications, and understanding the factors that limit mobility is essential for future devices. Yet, despite significant progress, mobilities in excess of the 2 x 10(5) cm(2) V-1 s(-1) demonstrated in free-standing graphene films(5,6) have not been duplicated in conventional graphene devices fabricated on substrates. Understanding the origins of this degradation is perhaps the main challenge facing graphene device research. Experiments that probe carrier scattering in devices are often indirect7, relying on the predictions of a specific model for scattering, such as random charged impurities in the substrate(8-10). Here, we describe model-independent, atomic-scale transport measurements that show that scattering at two key defects-surface steps and changes in layer thickness-seriously degrades transport in epitaxial graphene films on SiC. These measurements demonstrate the strong impact of atomic-scale substrate features on graphene performance.