The study of the electronic absorption spectra of 4-aminoazobenzene subjected to an external electric field in nonpolar and polar solvents shows that the field-induced change in the absorption intensity is dominated by the solvent-solute interaction. Moreover, solvent can determine the sign of the change of the absorption intensity. These experimental observations are supported by ab initio electronic structure calculations and are rationalized by analytic theory. The results carry particular importance for the numerous fundamental and practical applications of electric fields to understanding and design of new materials and biological systems.