In this paper we use infinite linear programming to study Markov control processes in Borel spaces and the average cost criterion in the "unichain" and "multichain" cases. Under appropriate assumptions we show that in both cases the associated linear programs are solvable and that there is no duality gap. Moreover, conditions are given for minimizing (respectively, maximizing) sequences for the primal (respectively, dual) programs to converge to optimal solutions.