We successfully synthesized a novel Ta4+ self-doped Ta2O5 quantum dots and the self-doped Ta2O5 quantum dots modified cotton-like g-C3N4 nanosheet (TCN) by vapor hydrolysis reaction and calcining treatment. The as-prepared Ta2O5 showed excellent visible light response from 400 to 800 nm, mainly because the formation of Ta4+ species, which was confirmed by EPR and XPS technology. The estimated bandgap of the self-doped Ta2O5 was similar to 2.78 eV, which was much lower than that of the commercial Ta2O5 (similar to 4.0 eV), and the visible light photocatalytic activity of the self-doped Ta2O5 for hydrogen production was about 7.98 mu molg(-1) h(-1). The photocatalytic activity of TCN photocatalyst for hydrogen production and RhB degradation was highly enhanced under visible light irradiation compared with those of g-C3N4 and the self-doped Ta2O5, indicating the higher charge separation efficiency, which was further confirmed by the photocurrent transient response, electrochemical impendence spectra and PL emission spectra. And the hydrogen evolution rate and the degradation rate constant for RhB degradation of TCN photocatalyst could reach 624.99 mu molg(-1) h(-1) and 0.1512 min(-1), which were about 3.7 and 11.9 times higher than those of pure g-C3N4, and 78.29 and 283.78 times higher than those of the self-doped Ta2O5, much higher than those of the simple physical mixture. The active species trapping experiment of TCN photocatalyst showed that RhB degradation was mainly attributed to the direct hole oxidation. Besides, the XPS results further confirmed that Ta-O-C chemical bond was formed between the as-prepared Ta2O5 and cotton-like g-C3N4. It was the strong synergistic interactions of Ta-O-C chemical bond and Ta4+ defect energy level that highly enhanced the photocatalytic activities of TCN catalyst for the hydrogen production and RhB degradation. (C) 2016 Elsevier B.V. All rights reserved.