Carbon-based nanomaterials, especially graphene, are considered to be the most hopeful alternatives to silicon in the near future of pushing to its limits. However, the zero band gap of graphene suppresses its application, it needs to develop a convenient method to modulate its Fermi level and band gap. Herein, we report a general approach to regulate Fermi level of graphene precisely by designed azobenzene molecules with different dipole moment and dipole orientation. The Raman spectra results demonstrate the p-p interaction between the azobenzene molecules and graphene, which result in the modulation of the Fermi level of graphene. Besides, based on the field effect transistor characteristic measurement, we also observe the Fermi level adjustment for the hole/electron doping to graphene from the spontaneous polarization effect of the azobenzene molecules, further speculating the regulation of electronic structure and providing a new route for changing the electrical properties of graphene.