A novel adsorbent, chitosan/cellulose biocomposite sponge (CCS) for removing Hg(II) ions rapidly and selectively from aqueous solution is successfully fabricated via a facile glutaraldehyde cross-linking reaction and lyophilization. The CCS can be rapidly separated and easily regenerated by simple extrusion compared with most particulate or powder absorbent. In the adsorption tests, the experimental data follows the pseudo-second-order kinetic model and Langmuir isotherm. The CCS uptakes more than 97% of the ultimate adsorption capacity in aqueous solution within 2 min, and the adsorption capacity is derived to be 495 mg/g. Furthermore, the CCS shows good selectivity for removing Hg(II) ions, and the coefficient of selectivity to Hg(II) ions is 461, 227, and 35 times higher than that to Cu(II), Pb(II), and Cd(II) ions, respectively. The mechanism for selective adsorption is elucidated via Fourier transform infrared spectrum (FTIR) and X-ray photoelectron spectroscopy (XPS) spectral analysis before and after adsorption. Density functional theory (DFT) calculations are also performed. Notably, the adsorption capacity retains 85% of its initial value (495 mg/g) after five cycles, indicating excellent reusability of the CCS. Significantly, the applicability of the CCS is evaluated in simulated Hg(II) effluent (PVC production). The results indicate that the removal efficiency of the CCS exceeds 84% of the initial value (11.45 ppm) within just 2 min. The sponge is a portable and eco-friendly composite, thereby has a great potential to decontaminate Hg(II) ions in water treatment.