Adhesion processes at the cellular scale are dominated by carbohydrate interactions, including the attachment and invasion of pathogens. Carbohydrate-presenting responsive polymers can bind pathogens and inhibit pathogen invasion by remote stimuli for the development of new antibiotic strategies. In this work, the adhesion forces of E. coli to monolayers composed of mannose-functionalized microgels with thermosensitive poly(N-isopropylacrylamide) (PNIPAM) and poly(oligo(ethylene glycol)) (PEG) networks are quantified using single-cell force spectroscopy (SCFS). 'When exceeding the microgels' lower critical solution temperature (LCST), the adhesion increases up to 2.5-fold depending on the polymer backbone and the mannose density. For similar mannose densities, the softer PNIPAM microgels show a significantly stronger adhesion increase when crossing the LCST as compared to the stiffer PEG microgels. This is explained by a stronger shift in swelling, mannose density, and surface roughness of the softer gels when crossing the LCST. When using nonbinding galactose instead of mannose, or when inhibiting bacterial receptors, a certain level of adhesion remains, indicating that also polymer-fimbria entanglements contribute to adhesion. The presented quantitative analysis provides insights into carbohydrate-mediated bacterial adhesion and the relation to material properties and shows the prospects and limitations of interactive polymer materials to control the attachment of bacteria.