A new composite photocatalyst CdSe-sensitized TiO2 nanorod array (CdSe/TiO2 NRA) grown on fluorine-doped tin oxide (FTO) coated with a TiO2:Yb3+,Er3+ thin film was constructed. The TiO2:Yb3+,Er3+ thin film acted as a medium for converting near-IR light to visible light via an upconversion process and reduced electron-hole recombination. TiO2 NRAs with high specific surface areas and well-aligned nanostructures can provide a fast transfer pathway for photogenerated electrons. The CdSe/TiO2 NRAs extended the optical response from the ultraviolet to visible region, which can generate more electron-hole pairs. The composite photocatalyst TiO2 NRAs/TiO2:Yb3+,Er3+ exhibited excellent photocatalytic activity toward the degradation of Rhodamine B. The composite structure can not only extend the absorption of TiO2 but also consequently reduce electron-hole recombination, which improve the photocatalytic efficiency of the composite photocatalyst. Moreover, the photocatalyst grown on FTO substrates directly makes it possible to collect and recycle.