We have developed a new photochromic material containing a stilbene moiety and a urea moiety based on cooperative effect of photoisomerization and proton transfer. An anthracene-urea-stilbene compound (stAn) exhibited cis-to-trans one-way photoisomerization via the excited triplet state. Furthermore, we succeeded in the switching from one-way to mutual photoisomerizations by the addition of a fluoride ion, which causes the ground state intermolecular proton transfer of stAn leading to negligible intersystem crossing from the excited singlet state.