A novel magnetic Fe3O4 modified reduced graphene oxide nanocomposite (Fe3O4@SiO2-rGO) was prepared by a covalent bonding method. The morphology and properties of the Fe3O4@SiO2-rGO were characterized by transmission electron microscopy and X-ray diffraction. The prepared Fe3O4@SiO2-rGO was tested as an efficient adsorbent for the removal of some dyes from aqueous solution for the first time. The performance of Fe3O4@SiO2-rGO was evaluated using methylene blue and neutral red as model compounds. Experiments were carried out to investigate the adsorption kinetics and adsorption capacity of the adsorbent and the effect of the adsorbent dosage and sample solution pH on the removal of the dyes. Kinetic data were well fitted by pseudo second-order model. The Langmuir model and the Freundlich model were used to study the adsorption isotherms. The Fe3O4@SiO2-rGO nanocomposite showed to be a highly efficient adsorbent with the advantage of separation convenience. The thermodynamic parameters indicated that the adsorption of the dyes onto the Fe3O4@SiO2-rGO was a spontaneous process.