The specific capacitance of biological membranes is a key physical parameter in bioelectricity that also provides valuable physicochemical information on composition, phase, or hydration properties. Cholesterol is known to modulate the physicochemical properties of biomembranes, but its effect on the specific capacitance has not been fully established yet. Here we use the high spatial resolution capabilities of in-liquid scanning dielectric microscopy in force detection mode to directly demonstrate that DOPC bilayer patches at 50% cholesterol concentration show a strong reduction of their specific capacitance with respect to pure DOPC bilayer patches. The reduction observed (similar to 35%) cannot be explained by the small increase in bilayer thickness (similar to 16%). We suggest that the reduction of the specific capacitance might be due to the dehydration of the polar head groups caused by the insertion of cholesterol molecules in the bilayer. The results reported confirm the potential of in-liquid SDM to study the electrical and physicochemical properties of lipid bilayers at very small scales (down to similar to 200 nm here), with implications in fields such as biophysics, bioelectricity, biochemistry, and biosensing.