Cellular membranes mediate vital cellular processes by being subject to curvature and transmembrane electrical potentials. Here we build upon the existing theory for flexoelectricity in liquid crystals to quantify the coupling between lipid bilayer curvature and membrane potentials. Using molecular dynamics simulations, we show that headgroup dipole moments, the lateral pressure profile across the bilayer, and spontaneous curvature all systematically change with increasing membrane potentials. In particular, there is a linear dependence between the bending moment (the product of bending rigidity and spontaneous curvature) and the applied membrane potentials. We show that biologically relevant membrane potentials can induce biologically relevant curvatures corresponding to radii of around 500 nm. The implications of flexoelectricity in lipid bilayers are thus likely to be of considerable consequence both in biology and in model lipid bilayer systems.