Hydrogels that are pH-sensitive and partially cross-linked by cobalt ferrite nanoparticles exhibit remarkable remanent magnetization behavior. The magnetic fields measured outside our thin disks of ferrogel are weak, but in the steady state, the field dependence on the magnetic content of the gels and the measurement geometry is as expected from theory. In contrast, the time-dependent behavior is surprisingly complicated. During swelling, the remanent field first rapidly increases and then slowly decreases. We ascribe the swelling-induced field enhancement to a change in the average orientation of magnetic dipolar structures, while the subsequent field drop is due to the decreasing concentration of nanoparticles. During shrinking, the field exhibits a much weaker time dependence that does not mirror the values found during swelling. These observations provide original new evidence for the markedly different spatial profiles of the pH during swelling and shrinking of hydrogels.