The polarization switching mechanism is used in various devices such as pyroelectric sensors and memory devices. The change in polarization mostly occurs by ion displacement. The development of materials whose polarization switches via electron transfer in order to enhance operation speed is a challenge. We devised a synthetic and crystal engineering strategy that enables the selective synthesis of a [CrCo] heterometallic dinuclear complex with a polar crystal structure, wherein polarization changes stem from intramolecular charge transfer between co and the ligand. Polarization can be modulated both by visible-light irradiation and temperature change. The introduction of chiral ligands was paramount to the successful polarization switching in the valence tautomeric compound. Mixing Cr and Co complexes with enantiopure chiral ligands resulted in the selective formation of only pseudosymmetric [CrCo] heterometallic complexes. Furthermore, the left-handed chiral ligands preferentially interacted with their right-handed counterparts, enabling molecules to form a polar crystal structure.