A class of decentralized tracking-type games is considered for large population multi-agent systems (MAS). The agents are described by stochastic discrete-time auto-regressive models with exogenous inputs (ARX models), and coupled together through their individual dynamics and performance indexes by terms of the unknown population state average (PSA). The performance index of each agent to minimize is a stochastic long term averaged group-tracking-type functional, in which there is a nonlinear term of the unknown PSA. The control law is decentralized and implemented via the Nash certainty equivalence principle. By probability limit theory, under mild conditions it is shown that: (a) the estimate of the PSA is strongly consistent; (b) the closed-loop system is stable almost surely, and the stability is independent of the number N of agents; (c) the decentralized control law is an asymptotic Nash equilibrium almost surely or in probability according to the property of the nonlinear coupling function in the performance indexes. (C) 2007 Elsevier Ltd. All rights reserved.