This work focuses on optimal switching with constraints. Our motivation stems from utility maximization of an indivisible market. The dynamic programming approach is used; the value function is characterized as the unique viscosity solution of a quasi-variational inequality. The unbounded domain introduces new challenges. By studying the sample paths of the diffusion at the boundary, a sufficient condition for the continuity of the value function is provided, yielding the desired characterization. Not only are the results of this work applicable to the utility maximization problem, but also they can be used for general optimal switching problems with finite regimes.