We consider critically loaded single class queueing networks with infinite buffers in which arrival and service rates are state (i.e., queue length) dependent and may be dynamically controlled. An optimal rate control problem for such networks with an ergodic cost criterion is studied. It is shown that the value function (i.e., optimum value of the cost) of the rate control problem for the network converges, under a suitable heavy traffic scaling limit, to that of an ergodic control problem for certain controlled reflected diffusions. Furthermore, we show that near optimal controls for limit diffusion models can be used to construct asymptotically near optimal rate control policies for the underlying physical queueing networks. The expected cost per unit time criterion studied here is given in terms of an unbounded holding cost and a linear control cost ("cost for effort"). Time asymptotics of a related uncontrolled model are studied as well. We establish convergence of invariant measures of scaled queue length processes to that of the limit reflecting diffusions. Our proofs rely on infinite time horizon stability estimates that are uniform in control and the heavy traffic parameter, for the scaled queue length processes. Another key ingredient, and a result of independent interest, in the proof of convergence of value functions is the existence of continuous near optimal feedback controls for the diffusion control model.