This paper presents an asymptotic analysis of hierarchical production planning in a general manufacturing system consisting of a network of unreliable machines producing a variety of products. The concept of a dynamic job shop is introduced by interpreting the system as a directed graph, and the structure of the system dynamics is characterized for its use in the asymptotic analysis. The optimal control problem for the system is a state-constrained problem, since the number of parts in any buffer between any two machines must remain nonnegative. A limiting problem is introduced in which the stochastic machine capacities are replaced by corresponding equilibrium mean capacities, as the rate of change in machine states approaches infinity. The value function of the original problem is shown to converge to that of the limiting problem, and the convergence rate is obtained. Furthermore, near-optimal controls for the original problem are constructed from near-optimal controls of the limiting problem, and an error estimate is obtained on the near optimality of the constructed controls.