We fabricated a novel reduced graphene oxide (rGO) and carboxyl functionalized single-walled carbon nanotube (SWNT-COON) nanocomposite based on all-inorganic architectures by vacuum filtration. The carboxyls at both ends of the SWNTs-COOH are cross-linked with the hydroxyls on the GO nanosheet surface via esterification reaction to improve the mechanical properties of the nanocomposite, which is superior to other common binary GO-based nanocomposites reported so far. Furthermore, the GO/SWNT-COOH nanocomposite shows high rejection rates for Evans blue (>99.5%) and methyl orange (>98%), while exhibiting high water flux. Thus, the fabricated nanocomposite, combined with high mechanical properties and dye separation performance, shows a promising opportunity in the future for designing and constructing novel multifunctional artificial nacre applied in water treatment.