Efficient xylose utilization will facilitate microbial conversion of lignocellulosic sugar mixtures into valuable products. In Escherichia coli, xylose catabolism is controlled by carbon catabolite repression (CCR). However, in E. coli such as the succinate-producing strain KJ122 with disrupted CCR, xylose utilization is still inhibited under fermentative conditions. To probe the underlying genetic mechanisms inhibiting xylose utilization, we evolved KJ122 to enhance its xylose fermentation abilities in parallel and characterized the potential convergent genetic changes shared by multiple independently evolved strains. Whole-genome sequencing revealed that convergent mutations occurred in the galactose regulon during adaptive laboratory evolution potentially decreasing the transcriptional level or the activity of GalP, a galactose permease. We showed that deletion of galP increased xylose utilization in both KJ122 and wild-type E. coli, demonstrating a common repressive role of GalP for xylose fermentation. Concomitantly, induced expression of galP from a plasmid repressed xylose fermentation. Transcriptome analysis using RNA sequencing indicates that galP inactivation increases transcription levels of many catabolic genes for secondary sugars including xylose and arabinose. The repressive role of GalP for fermenting secondary sugars in E. coli suggests that utilization of GalP as a substitute glucose transporter is undesirable for conversion of lignocellulosic sugar mixtures.