Fluorescence intermittency and excited-state decay measurements are carried out on single CdSe-CdS core-shell quantum dots (QD) stabilized with chalcogenidometalates (ChaMs, In2Se42-, or Sn2S64-)(-). The results are used to probe the nature and distribution of charge trapping sites in the QD local environment. A comparison is made between capping by a neutral organic ligand (oleylamine) and a small inorganic ligand with high charge density (ChaMs). Overall, shorter on-state durations and longer off-state durations are observed for the ChaMs. These results indicate an increased density of charge trapping sites and increased stabilization of surface-trapped charges. By varying the thickness of the CdS shell, we identified hole trapping by the ligand as the dominant charging mechanism in ChaM-capped QDs. Faster excited-state decay rates are measured for the ChaM-capped QDs, highlighting the role of strongly stabilized trapped charges in this system. Using cyclic voltammetry measurements of the ChaMs, an energy level diagram is constructed relating the ChaMs and CdSe-CdS-QDs that explains their superior performance as active layers in photodetectors.