Forces between colloidal particles determine the performances of many industrial processes and products. Colloidal force measurements conducted between a colloidal particle AFM probe and particles immobilized on a flat substrate are valuable in selecting appropriate surfactants for colloidal stabilization. One of the features of inorganic fillers and extenders is the prevalence of rough surfaces even the polymer latex particles, often used as model colloidal systems including the current study, have rough surfaces albeit at a much smaller scale. Surface roughness is frequently cited as the reason for disparity between experimental observations and theoretical treatment but seldom verified by direct evidence. This work reports the effect of nanoscale surface roughness on colloidal force measurements carried out in the presence of surfactants. We applied a heating method to reduce the mean surface roughness of commercial latex particles from 30 to 1 nm. We conducted force measurements using the two types of particles at various salt and surfactant concentrations. The surfactants used were pentaethylene glycol monododecyl ether, Pluronic F108, and a styrene/acrylic copolymer, Joncryl 60. In the absence of the surfactant, nanometer surface roughness affects colloidal forces only in high salt conditions when the Debye length becomes smaller than the surface roughness. The adhesion is stronger between colloids with higher surface roughness and requires a higher surfactant concentration to be eliminated. The effect of surface roughness on colloidal forces was also investigated as a function of the adsorbed surfactant layer structure characterized by AFM indentation and dynamic light scattering. We found that when the layer thickness exceeds the surface roughness, the colloidal adhesion is less influenced by surfactant concentration variation. This study demonstrates that surface roughness at the nanoscale can influence colloidal forces significantly and should be taken into account in colloidal dispersion formulations.