We numerically investigate the effects of the particle surface roughness on the rheological properties of dense suspension of non-Brownian rough particles. We use the Ball-Melrose approximation to calculate the hydrodynamic interactions and an elastic-plastic monoasperity model with normal load dependent coefficient of friction to model the contact dynamics and the friction between the particles. We successfully reproduce the shear thinning behavior in non-Brownian suspensions, which is often observed in experiments. The relative viscosity and the normal stress difference increase with the roughness of the particles. These findings show satisfactory agreement with recent experiments. We also fit our data to the Maron-Pierce law to predict the relative viscosity with varying volume fractions and roughness. The jamming volume fraction decreases with the particle roughness owing to the increase in effective particle radii and the average coefficient of friction with roughness. The jamming fraction is also dependent on the stress and increases with stress. This directly leads to an increase in the relative viscosity with roughness in suspensions of rough non-Brownian particles. These findings suggest that accurate modeling of the contact dynamics and friction is crucial to accurately simulate the rheological behavior of dense suspensions subjected to shear flow.