Perylene imides have been an object of research for 100 years and their derivatives are key n-type semiconductors in the field of organic electronics. While perylene diimides have been applied in many electronic and photonic devices, their use can be traced back to the first efficient organic solar cell. By functionalizing different positions of the in total 12 positions (four peri, four bay, and four ortho-positions) on the perylene core, perylene imides with significantly different optical, electronic and morphological properties may be prepared. Perylene imides and their derivatives have been used in several types of organic photovoltaics, including flat-, and bulk-heterojunction devices as well as dye-sensitized solar cells. Additionally perylene imides-based copolymers or oligomers play an important role in single junction devices. In this review, the relationship between the photovoltaic performance and the structure of perylene imides is discussed.