The chemical instability of palladium-based metal membranes is the major hurdle for their commercial applications in gas separation and membrane reactor processes. When the metal membranes are exposed to a gas stream consisting of sulfur- or carbon-containing gases or even just steam, a drop in permeability or selectivity may occur depending on the composition of the metal membrane, and the extent of the decrease depends on the temperature and partial pressure. This paper provides a comprehensive review and in-depth analysis of the poisoning effects and mechanisms of sulfur- or carbon-containing compounds and H2O on Pd-based metal membranes. The paper also summarizes several methods reported in the literature for improving the chemical stability of Pd-based metal membranes with an analysis on the underlining principles for chemical stability improvement. These methods include alloying Pd with other metals (such as Cu, Ni, Fe, Pt, and Ag) preferably in the body-centered-cubic crystal structure, preparing nanostructured or amorphous Pd alloy thin membranes, and modifying the surface of a Pd-based membrane by a metal. These methods have improved the chemical stability of the Pd-based membranes to some extent, and more studies in this direction are still needed in order to develop chemically stable metal membranes for various industrial applications.