Membrane dipole potential influences a variety of important biological processes involving cell membranes. Because it is quite challenging to directly measure the membrane dipole potential in experiments, molecular dynamics (MD) simulation has emerged as a powerful tool for a reasonable prediction of the dipole potential. Although MD predictions agree well with experiments about the sign of the dipole potential, the magnitude of the dipole potential varies significantly with the force field parameters. It has been shown that the positive dipole potential of phosphatidylcholine (PC) bilayer membranes would be overestimated by a nonpolarizable model owing to the treatment of many-body polarization effects in a mean-field fashion. In this work, we carried out atomistic MD simulations of the diphytanyl PC (ether-DPhPC) and diphytanoyl PC (ester-DPhPC) bilayers and made a comparative study of three different nonpolarizable water models (TIP3P, TIP4P, and TIPSP). Interestingly, we discover that the calculated dipole potential by the TIPSP model shows good agreement with the result obtained using the cryoelectron microscopy experiment, suggesting that a better description of electrostatic interactions in a nonpolarizable water model can effectively ameliorate the overestimation in the calculation of the dipole potential. In addition, our MD results show that the substitution of the ether linkage for the ester linkage of phospholipid bilayers would bring about a change in the orientation of the linkage group with respect to the bilayer normal, leading to the difference in the membrane dipole potential. Surprisingly, although water molecules provide a major contribution to the positive dipole potential, they have a limited impact on the difference of the dipole potential between the ether-DPhPC and ester-DPhPC bilayer membranes.