In this work, multifunctional hydrogels with vivid color change and shrinking-swelling response to temperature, ion strength, and alternating magnetic field are fabricated via magnetic assembly. The hydrogels show gradual shift colors from yellowish green to green, cyan, blue, purple, and even reddish violet in response to temperature or ion strength. In the response process, the whole color modulation process is fully reversible and transferable along with a relative short response time. Especially, the magnetism and porous structure of the hybrid hydrogel enable it to be a potential carrier for hydrophobic molecules. Taking advantage of the magnetocaloric responsiveness, the dyed oil loaded hydrogel exhibits a controllable release behavior in each reversible shrinking-swelling cycle under an alternating magnetic field. This multi-responsive hydrogel can hold promise for practical engineering applications, including sensors, displays, and controlled release.