A robust skin color detection method using adaptive skin color modeling on YES color space is proposed. The skin colors are classified into three skin regions according to the Y value, and each region is modeled as the skin color in ES plane. By introducing an iterative estimation method of the illuminant color, we effectively remove the pseudo-skin colors that have been erroneously detected in skin color using the standard deviation of E-histogram. We also improve the skin color detection rate using a post-processing expansion mask that can include highlight and shadow pixels near the skin region as the skin color region.