We perform grand-canonical molecular simulations to study the molecular mechanism of clay swelling hysteresis as a function of the relative humidity. In particular, we focus on the transition from the one- to the two-layer hydrate and the influence of three types of counterions (Li+, Na+, and K+). Our results cover the experimental relative humidity region where swelling and shrinking usually occur. We show that the thermodynamic origin of swelling hysteresis is a free-energy barrier separating the layered hydrates. This free-energy barrier is dominated by breaking and formation of hydrogen bonds between and within water layers. This network of water molecules is similar for all counterions, but the positions of these counterions depend upon their size. The relatively large K+ counterions show more affinity for clay surface adsorption, which increases the free-energy barrier and inhibits swelling. On the other hand, the relatively small Li+ counterions are quite well-accommodated in the water network, and thereby, they can form a new swelling state with a basal spacing of approximately 13.5 angstrom. This new swelling state is an alternative explanation for the widely accepted simultaneous occurrence of two or more swelling phases.