A non-Markovian generalization of the Chapman-Kolmogorov transition equation for continuous time random processes governed by a waiting time distribution is investigated. It is shown under which conditions a long-tailed waiting time distribution with a diverging characteristic waiting time leads to a fractional generalization of the Klein-Kramers equation. From the latter equation a fractional Rayleigh equation and a fractional Fokker-Planck equation are deduced. These equations are characterized by a slow, nonexponential relaxation of the modes toward the Gibbs-Boltzmann and the Maxwell thermal equilibrium distributions. The derivation sheds some light on the physical origin of the generalized diffusion and friction constants appearing in the fractional Fokker-Planck equation.