The microstructure evolution of crystalline MgB2 layer is examined during ex situ annealing of the evaporated amorphous boron (a-B) with the Mg vapor. It is identified that an MgB2 layer starts to form at the surface by the nucleation and growth process in polycrystalline form. It appears that there exist two distinct growth fronts at both sides of MgB2 layer. Consequently, the further growth of MgB2 layer proceeds by both the out-diffusion of boron through the MgB2 layer and by the in-diffusion of Mg through the MgB2 layer. The microstructural evolution of this layer shows significant difference depending on the location of these two growth fronts. In addition, we also observed the microstructual. variation of ex situ annealed MgB2 layer as a function of the composition of as-deposited film. When the composition of as-deposited film is close to MgB2, the samples show the smallest surface roughness and best superconducting properties.