Extremum seeking control (ESC) is an adaptive control scheme that locates an extremum of an input-output map, without any explicit knowledge of this map apart from its existence. As is typical in adaptive control an integrator is used to drive the parameters that are being adapted. Due to this integrator, it is possible that the adapted parameters wander outside their physically relevant domain as the underlying adaptation technique is blind to this constraint. As these constraints may represent realistic operational limits it is important to design ESC to respect them. Two such ESC schemes are proposed. One is based on a constrained optimization approach in which some penalty function is used to adapt the search so as not to violate the constraints. The other technique uses an anti-windup scheme, a widely used mechanism in engineering to prevent windup of integral action in a controller. It is demonstrated that both methods are essentially equivalent. In that for any penalty-function based ESC, there exists an equivalent anti-windup ESC whose phase portrait is a global approximate of the penalty-function ESC. Some simulations are presented to illustrate the main results.