The sustainability of global dean and safe water supply is one of the grand challenges facing the world. Membrane technology based on polymeric membranes is one of the most important and widely recognized technologies for desalination and wastewater treatment. While polymeric membranes are known to be plagued with some bottlenecks, the technical progress and the accompanying knowledge in inorganic membrane development have grown inexorably to solve some of the underlying issues. Aside from the conventionally used ceramic membranes which based on metal oxides, nanostructures such as zeolites, metal organic frameworks and carbon based materials have sparked enormous interest in the preparation of inorganic membranes owing to their tunable nanoscaled structural properties that can render excellent rejection and/or ultrafast water transport. This review provides insights into the physico-chemical properties and fabrication approaches of different classes of inorganic membranes. The transport mechanisms that are associated to their unique structural features are also discussed. Furthermore, the performance evaluation of these inorganic membranes in a wide spectrum of desalination and wastewater treatment applications are also elaborated. Finally, the challenges in the development of inorganic membrane for practical commercial application are identified and the future perspectives are presented.