Small polymer latex particles in the size range 15 nm < R < 100 nm with different types of covalently bound surface stabilizing groups are made via emulsion polymerization using diverse functionalization techniques like incorporation of polar comonomers, surface-active initiators, or polymerizable surfactants. These colloidal dispersions in water are swollen with toluene in the absense of free surfactants until equilibrium swelling is reached. The equilibrium size of the swollen latices is determined by combining densitometry and dynamic light scattering. For all surface groups, we observe a pronounced dependence of the swelling ratio on particle size where absolute values of swelling are much lower than described by classical theories. A modified description is presented which considers size-relevant effects (such as the Kelvin pressure and depletion) by an additional osmotic pressure term which increases with the inverse of the particle size. Other classical effects of colloids such as a size dependence of the thermodynamic interaction parameter or the inter face energy can be excluded by quantitative considerations. The presented modified swelling equation describes the experimental data well and is expected to have some broader relevance for swelling phenomena of all colloidal systems. Charge stabilization as compared to steric stabilization results in higher swelling ratios and significant lower values for the interface energy. This underlines the importance of coulombic repulsion between fixed charges on the same particle for the swelling process as well as the existence of a "surface anchoring" effect, both of which are not considered in current concepts of the swelling process, too.