pH and thermoresponsive polymers have been prepared by copolymerizing N-isopropylacrylamide (NIPAM) with various amounts of ionizable comonomers, either acrylic acid (AA) or N-[3-(dimethylamino)propyl]methacrylamide (MADAP). In aqueous solution, the LCST-type phase transition of these copolymers studied by differential scanning calorimetry is strongly influenced by the comonomer ratio. Under un-ionized conditions, the phase transition temperature progressively increases with MADAP content at pH 12 while it remains un-changed or slightly decreases with AA at pH 3 due to the formation of hydrogen bonds between AA and NIPAM units. When the copolymer chains are progressively charged by tuning the pH, the phase transition of PNIPAM-AA and PNIPAM-MADAP is shifted at higher temperature and is no longer observable below 60 degrees C when the ionic content exceeds 10%. By comparison with these single systems, where the association properties can be finely adjusted by coupling hydrophobic attractions and electrostatic repulsions, we also investigate the possibility to couple hydrophobic interactions with electrostatic attractions by mixing oppositely charged copolymers: PNIPAM-AA and PNIPAM-MADAP. This study was carried out with the copolymer pair containing 10 mol % of ionizable groups (A10 and M10) which was the most adaptable one from the point of view of responsivity. At pH 7, when AA and MADAP units are ionized, the copolymer chains are separately soluble in water in the whole temperature range while their mixture, also soluble at room temperature, phase separates upon heating. The original feature highlighted in this work is that the phase transition proceeds through a selective mechanism between complementary chains (formation of a reversible interpolyelectrolyte complex) and that this selectivity can be switched with the pH. Indeed, starting at room temperature with the copolymer mixture (A10 and M10), which remains homogeneous at all pH, we demonstrate that the association process can be switched by increasing the temperature and that the pH can be used to specifically address these associations: from A10/A10 at low pH to A10/M10 at pH 7 up to M10/M10 at high pH. Finally, the responsive precursors A10 were grafted onto a polyacrylamide backbone, and the viscoelastic properties of graft copolymers were studied in the semidilute regime and compared with calorimetric data. In this way, we show that the same set of interactions can be readily applied to more sophisticated macromolecular assemblies with the responsive formation of physical gels under pH and temperature control.