We study the deformation of "filled" (with a neutral polymer) polyelectrolyte microcapsules under applied load using an atomic force microscope (A-FM)-related force-measuring device. The experimental setup allows combining dynamic force measurements with high-resolution optics. The "filled" capsules are prepared from "hollow" ones (water inside) by changes in their permeability for a high molecular weight polymer by adding organic solvent. At low applied load, capsule deformation was found to be elastic, reversible, and independent of the concentration of the inner polymer, being entirely controlled by the shell properties. Above a certain load capsules deform substantially and partly irreversibly. They start to show variability in the behavior and dependence of their deformation profiles on the driving speed and concentration of the inner polymer solution, which is likely due to the enhanced permeability of the stretched shell. The "filled" capsules were found to be always several times softer than the preformed "hollow" ones, which is shown to be a consequence of the treatment of the shell by organic solvent.