Reflectance difference spectra of anion-covered GaAs (001) and (110) surfaces are calculated. By varying atom binds of top-layer anions from N to P, As, and Sb, the spectral peak shifts toward lower energy, sharpens, and becomes large, being in good agreement with experiments. It is shown that these changes are closely related to the localization nature of anion dangling bond states at the surface and reflect the atomic orbital energies of anion atoms.