Adhesion strength of surface-enhanced Raman scattering (SERS) substrates is an important factor for practical applications. Ag nanorods (AgNRs) based SEAS sensors possess many outstanding merits, while the film adhesion is very weak. To strengthen the bonding of AgNRs on supporting bases, two effective approaches of adding transition layers and subsequent thermal annealing were utilized. Before AgNRs deposition, a Ti intermediate layer was deposited onto silicon wafers to relieve the large internal stress caused by the physical property differences between Ag and Si films. An ultrathin Al2O3 layer was then coated onto AgNRs surfaces, which enhanced both the thermal stability and mechanical robustness of the Ti-AgNRs substrate. Thereafter, TiAgNRs@Al2O3 was annealed to further facilitate the atomic diffusion and interaction between Ag, Ti and Si films. The optimized substrate exhibited a much improved scratch resistance property, and could sustain long-term ultrasonic vibration measurement. The excellent adhesive property ensures the stability of SEAS substrates during transportation, storage and employment. Such a structure design leads to robust and reliable SEAS sensors for real-life applications, and provides a valuable guidance for the adhesion enhancement of thin films.