Infrared molecular beam depletion spectroscopy has been employed to study the vibrational spectroscopy of water molecules and small water polymers [(H2O)(n), n=2,3,4] embedded in large liquid helium clusters (He-N, 100 less than or equal to(N)less than or equal to 5000). The spectral region between 3500 and 3800 cm(-1) was covered with an injection-seeded optical parametric oscillator. The following vibrational bands could be located and investigated : for the monomer the v(3) asymmetric stretch, for the dimer the asymmetric stretch of the proton acceptor molecule and the free and bonded O-H stretches of the donor molecule, for the trimer both free and bonded O-H stretches, and for the tetramer the free O-H stretch. By comparison with the data on foe gas phase water complexes, it was found that the helium host clusters induce only minor perturbations in form of small frequency redshifts and that they constitute an ideal nano-matrix. Furthermore, it was found that both the water molecule and the dimer rotate almost freely in the host cluster and that the internal-rotationlike tunneling motion of the water dimer is not quenched. Due to the extremely low internal temperatures, a splitting of the trimer band associated with the O-H ring vibration could be resolved for the first time. This splitting indicates that the trimer structure is asymmetric, as has been predicted by theoretical calculations.