The structural changes as functions of spinning conditions and heat treatments were investigated with respect to the structural feature of PAN-based carbon fibers by scanning tunneling microscopy (STM). The distinct granule structure on the cross section of both high tensile strength and high modulus carbon fibers was observed by SEM, while slender granule-shape domain on the longitudinal surface was revealed by STM. A structure model was proposed, which depicted that the PAN-based carbon fiber was a heterogeneous structure composed of aggregated mesostructural domains. These domains were closely arranged into spiral form along fiber axis, allowing the fibers have high strength and good elongation. The initial shape and size of domains was determined by the precursor composition and spinning conditions and also strongly depended on the heat-treated temperature and stretching conditions. The smaller or slender domain, the higher tensile strength obtained for fibers. We expect that the PAN-based carbon fiber with better performance should be produced by optimizing the size and shape of these domains.