In the past few decades, natural gas has attracted worldwide attention as one of the most desired energy sources owing to its more efficient and cleaner combustion process compared to that of coal and crude oil. Due to the presence of impurities, raw natural gas needs to be upgraded to meet the pipeline specifications. Membrane-based separation is a promising alternative to conventional processes such as cryogenic distillation and pressure swing adsorption. Among the existing membranes for natural gas upgrading, polymeric membranes and inorganic membranes have been extensively explored, but each type has its own pros and cons. The development of mixed matrix membranes (MMMs) by incorporating organic/inorganic fillers into the polymer matrix provides a good strategy to combine the merits of each material and fabricate novel membranes with superior gas separation performance. In this review, we first discuss the recent advances in MMMs showing potentials in natural gas upgrading. Special attention is paid to a detailed evaluation on the polymer and filler choices for acidic gas removal. After that, we analyze factors that influence the membrane separation performance and summarize effective strategies reported in the open literature for the fabrication of high-performance MMMs. Finally, a perspective on future research directions in this field is presented.