Phase limitations of both continuous-time and discrete-time Zames-Falb multipliers and their relation with the Kalman conjecture are analyzed. A phase limitation for continuous-time multipliers given by Megretski is generalized and its applicability is clarified; its relation to the Kalman conjecture is illustrated with a classical example from the literature. It is demonstrated that there exist fourth-order plants where the existence of a suitable Zames-Falb multiplier can be discarded and for which simulations show unstable behavior. A novel phase limitation for discrete-time Zames-Falb multipliers is developed. Its application is demonstrated with a second-order counterexample to the Kalman conjecture. Finally, the discrete-time limitation is used to show that there can be no direct counterpart of the off-axis circle criterion in the discrete-time domain.