We demonstrate that polyelectrolyte multilayer film (PEM) systems constructed using layer-by-layer (LbL) assembly of poly(allylamine hydrochloride) (PAH) and poly(acrylic acid) (PAA) exhibit significant: swelling/deswelling hysteresis in humid air environments, swelling less upon absorption of water from a dry state than upon deswelling from a hydrated (approximate to 100% RH) state. The hysteretic swelling, measured utilizing in-situ reflectometry, is reproducible over multiple absorption-desorption cycles and is essentially absent in spin-coated films of either pure polyelectrolyte. Kinetic swelling measurements support the notion that the structural response to humidity in PEMs can occur over two distinct regimes: an initial swelling response on the order of seconds to minutes as water enters the film and a longer time scale structural relaxation on the order of hours to days. While the structural relaxation appears to influence the magnitude of the hysteresis effect, it cannot account for it entirely; rather, upon long-term storage at an intermediate humidity, the internal structure of a PEM retains some: "memory" of its humidity exposure history, similar to some biologically derived textile and food materials. We propose that the mechanism of hysteresis in PEM systems is tied to a combination of internal molecular restructuring in response to ambient moisture and the frustration of structural relaxations by the internal electrostatic cross-link network within the film. Our results suggest that factors such as the drying technique and the manner in which films are equilibrated to ambient humidity may subsequently affect PEM properties as well as the performance of these systems in applications such as environmental sensing.