Small van der Waals clusters of sulfur hexafluoride (SF6) and mixed SF6-rare gas clusters were prepared inside large droplets of helium-4, with each droplet consisting of about 4000 helium atoms. A diode laser was used to measure the high-resolution infrared spectra of these clusters in the vicinity of the nu(3) vibrational mode. In all cases rotational structure was observed, indicating that the embedded species rotate nearly freely, although they had been cooled to a temperature of 0.37 kelvin. The results indicate that helium droplets are probably superfluid and thereby provide a uniquely cold yet gentle matrix for high-resolution spectroscopy.