The microalloying of low-alloyed steels with Nb can improve the strength-to-toughness balance. Such an effect of Nb is usually ascribed to the refinement of the grain structure occurring in the austenite regime during hot forming. In the present work, we report that Nb enhances the impact toughness of a low-alloyed Cr-Mo steel by a mechanism which has not been appreciated so far. The lower impact toughness in the Nb-free Cr-Mo steel is due to segregation of Mo to boundaries, which facilitates the formation of fine Mo-rich xi-phase carbides lining up along the boundaries. This further promotes the nucleation and propagation of microcracks. The addition of Nb leads to the formation of Mo-enriched NbC particles. The interfaces between these particles and the matrix supply new preferential sites for precipitation of Mo-rich xi-phase carbides upon subsequent tempering. In this way, Nb additions result in a decrease of Mo segregation to boundaries, significantly reducing the precipitation of xi-phase carbides on grain boundaries, thus leading to improved impact toughness. In addition to the classical microstructural explanation (grain size effect), this chemical role of Nb sheds new light on the design strategies of advanced low-alloyed steels with optimized strength-to-toughness ratios.