The adsorption gas of coal matrix will cause its swelling, change the fracture structure, and further affect the permeability of the coal seam. In this study, CT image processing technology was used to numerically simulate the swelling of different extents (0.05%, 0.10%, 0.15%, 0.20%, and 0.25%). At the same time, the evolutions of geometric parameters, connectivity, and fractal dimension of fractures under different swelling extents were quantified. The results show that with the increase of the swelling extents, the average length, width, and aperture of fractures decreased by 18.69%, 24.14%, and 52.24%, respectively. The connected porosity of fractures decreased by 44.83%, indicating fractures are closed and the connectivity becomes worse. It is noteworthy that when the swelling extent is less than 0.1%, the average aperture and connected porosity of fractures are significantly reduced by 32.84% and 31.03%, respectively. When it is greater than 0.15%, and the average aperture and width of fractures are clearly reduced by 49.25% and 23.28%, respectively. The fractal dimension increased from 2.132 to 2.211, indicating that the fracture structure became complex. Additionally, using Darcy's law to determine the change in absolute permeability during swelling, the results show that there was a significant negative exponential correlation between permeability and swelling extents. The decrease of aperture and connected porosity are the main factors of poor permeability.