Polymeric materials have been widely used for the shape-controlled synthesis of metallic nanoparticles. In addition to coordination between the polymers and the particle surfaces during seed growth, a few studies have reported the anisotropic growth as the result of polymer intervention solely during seed formation. However, this mechanism has not been thoroughly elucidated. In this work, we synthesize silver nanoplates (AgNPLs) and investigate the influence of two representative zwitterionic polymers, poly(2-methacryloyloxyethyl phosphorylcholine) (PMPC) and poly(sulfobetaine methacrylate) (PSBMA), on Ag seed formation. PMPC and PSBMA carry net zero global charges and non-zero local charges, which are ideal for investigating polymer-Ag+ interactions during the early stage of nanoparticle synthesis. The non-zwitterionic polymers carry different global charges, and they are examined in parallel with the zwitterionic polymers. X-ray photoelectron spectroscopy reveals that coordinate bonding between Ag+ and oxygen atoms in polymer functional groups strongly promotes the anisotropic growth of AgNPLs, even without the addition of polymers during growth. In contrast, nitrogen atoms in the polymers inhibit anisotropic growth. Our findings are further generalized in the context of seed-mediated growth by growing the seeds synthesized with the zwitterionic polymers at various concentrations, with various number average molecular weights, and at different temperatures.