The enhancement of wormlike micellar structure after the addition of O-carboxymethylchitosan (OCMCS) or hydrophobically modified O-carboxymethylchitosan (hm-OCMCS) has been studied by rheology and freeze-fracture transmission electron microscopy (FF-TEM). The results show that the viscoelastic properties of the wormlike micelles composed of Tween 80 and Brij 30 increase significantly after the addition of OCMCS. However, as the OCMCS molecules are hydrophobically modified, the viscosity enhancement is reduced and even diminishes with an increase of the length of aliphatic chains. FF-TEM was also used to provide a direct investigation of the microstructure changes of wormlike micelles after the addition of OCMCS and hm-OCMCS. Combined with the rheological behavior of OCMCS and hm-OCMCS, it is speculated that these results originate from a greater degree of intra-aggregation of polymers after hydrophobic modifications, which results in a weaker interaction between polymers and micelles. Additionally, the influence of temperature on the rheological behavior of polymer/wormlike micelle systems also demonstrates the intra-aggregation of polymers is the main impact factor in these systems.