To detect hydrogen leakage as soon as possible, researchers try their best to improve the sensitivity and response speed of the hydrogen sensor. However, the sensitivity and response speed are two contradictive parameters. It is hard to improve them simultaneously. The transmission optical fiber sensor with multi-layer films is the only structure which can increase the response speed and enhance sensibility simultaneously. However, because of its special structure, the output signal of the sensor often drifts. This paper designed an in-situ observation system to study the reason why the sensor drifts. The in-situ observation system found a periodic oscillation pattern for the transmission spectrum which depends on the wavelength of the light source. The transmission spectrum patterns of the sensor with multi-layer Palladium-Yttrium (Pd-Y) alloy films under different hydrogen concentrations were analyzed. The source of drift error induced by the wavelength shift of the light source was confirmed. By using a moving average algorithm, the error characteristics of the sensor were analyzed and simulated. The results show that the increased sweep width of the laser can effectively restrain the signal drift of sensors. Particularly, when the sweep width of the laser just is the integer multiples of the period of the transmission spectrum, the suppression of the oscillation was optimal. A sensor with a wavelength-swept laser was implemented. For the sweep width of 1.1 nm, the maximum wavelength sensitivity of the sensor is only 0.046 mv/pm. The wavelength drift error is significantly less than that without signal processing. The sensor has achieved a detection limit of 0.05% which is identical to the sensor with the frequency-stabilized laser. Finally, a design principle was proposed to optimize the light source parameters and structure parameters of the probe for the high stability of the optical fiber hydrogen sensor. (C) 2019 Hydrogen Energy Publications LLC. Published by Elsevier Ltd. All rights reserved.