Injection-molded poly(butylene terephthalate) (PBT) is generally annealed at a temperature above the operating temperature to remove residual internal stress and obtain high-dimensional stability. However, this annealing treatment causes poor adhesion property and affected quality and lifetime for the products. In this study, a plausible mechanism for why such problems are caused by the annealing treatment is proposed using additive free PBT. After PBT was annealed under given conditions, surface structure and adhesion strength with an epoxy adhesive were examined. Surface roughness of the PBT film monotonically increased with increasing annealing time, and the tensile adhesive strength concurrently decreased. In addition, the height difference between hill and valley of the surface after adhesive failure became larger, meaning that the failure took place in the PBT phase beneath the interface with the epoxy region. Based on these findings, it was claimed that the poor adhesion of PBT after the annealing treatment originated from a weak boundary layer, which was formed at the outermost surface of PBT. Finally, the presence of the weak boundary layer was directly confirmed by transmission electron microscopy.