In this paper, we have used deep Q-learning networks (DQN) to find a colored coating for solar cells with high transmission. A basic structure with a huge range of possibilities was given to DQN, and it was designed to find the best structures fitting our purpose. The number of possibilities given to the model was more than 12 billion. Our model could find the structures with higher transmission and deeper colors compared to other human researchers in around 32,000 steps. Our numerical results cover a large area of color gamut which can be used for aesthetic purposes.