Vacancy-ordered lead-free perovskites with more-stable crystalline structures have been intensively explored as the alternatives for resolving the toxic and long-term stability issues of lead halide perovskites (LHPs). The dispersive energy bands produced by the closely packed halide octahedral sublattice in these perovskites are meanwhile anticipated to facility the mobility of charge carriers. However, these perovskites suffer from unexpectedly poor charge carrier transport. To tackle this issue, we have employed the ultrafast, elemental-specific X-ray transient absorption (XTA) spectroscopy to directly probe the photoexcited electronic and structural dynamics of a prototypical vacancy-ordered lead-free perovskite (Cs3Bi2Br9). We have discovered that the photogenerated holes quickly self-trapped at Br centers, simultaneously distorting the local lattice structure, likely forming small polarons in the configuration of V-k center (Br-2(-) dimer). More significantly, we have found a surprisingly long-lived, structural distorted state with a lifetime of, similar to 59 mu s, which is similar to 3 orders of magnitude slower than that of the charge carrier recombination. Such long-lived structural distortion may produce a transient "background" under continuous light illumination, influencing the charge carrier transport along the lattice framework.