This paper presents a thorough investigation over characteristics and device performance of Cu2ZnSnSe4 (CZTSe) thin films and CZTSe solar cells prepared on Mo substrate, and on transparent oxide layers including SnO2:F (FTO) layer, and In2O3:Sn (ITO) layer. The resultant CZTSe thin film is characterized by X-ray diffraction (XRD), secondary ion mass spectroscopy (SIMS), field emission scanning electron microscope (FESEM), and photoluminescence (PL), and is subsequently fabricated into CZTSe solar cell. The characterization measurements have unveiled that, different substrate material can affect the diffusion of sodium and zinc, hence alters the distribution of ZnSe, surface morphology and crystalline structure of the CZTSe thin film. In comparison, FTO shows superior stability among Mo substrate and ITO substrate during high temperature fabrication process for CZTSe thin film solar cells. For weatherability test of the three CZTSe solar cells with different substrates, all of the three devices have also been exposed to damp heat damage. Test results shows the least efficiency decline on the CZTSe solar cell with FTO substrate after damp heat damage, suggesting that the CZTSe solar cell with FTO substrate is the most resistive to damp heat weathering. The CZTSe solar cell with Mo substrate can be deteriorated at Mo substrate layer and shows more drastic efficiency decrease. The comparison reflects a high potential in CZTSe solar cell for the next generation photovoltaic applications, such as see-through solar cells and building-integrated photovoltaic (BIPV).