Surface phonon-polaritons (SPhPs), collective excitations of photons coupled with phonons in polar crystals, enable strong light-matter interaction and numerous infrared nanophotonic applications. However, as the lattice vibrations are determined by the crystal structure, the dynamical control of SPhPs remains challenging. Here, we realize the all-optical, non-volatile, and reversible switching of SPhPs by controlling the structural phase of a phase-change material (PCM) employed as a switchable dielectric environment. We experimentally demonstrate optical switching of an ultrathin PCM film (down to 7 nm, < lambda/1,200) with single laser pulses and detect ultra-confined SPhPs (polariton wavevector k(p) > 70k(0), k(0) = 2 pi/lambda) in quartz. Our proof of concept allows the preparation of all-dielectric, rewritable SPhP resonators without the need for complex fabrication methods. With optimized materials and parallelized optical addressing we foresee application potential for switchable infrared nanophotonic elements, for example, imaging elements such as superlenses and hyperlenses, as well as reconfigurable metasurfaces and sensors.