The paper presents a study of several problems related to spectral factorizations. We assume only a very weak form of coercivity for the p x p spectral function Phi and look at the set W-m of all rectangular, p x m spectral factors. The main object is the arithmetization of the geometry of the set of minimal, stable spectral factors by employing Hardy space techniques and the arithmetic of inner functions. Particular attention is paid to the study of various partial orders associated with the set of spectral factors.