Convergence of iterative dynamic programming (IDP), employing piecewise linear control to establish the optimal control policy of very high dimensional smooth systems, was examined by considering two linear systems with quadratic performance indices. The convergence of IDP was found to be systematic and no problems were encountered in determining the optimal control of a system having 250 state variables and 250 control variables. Computationally, the use of IDP for the 250 dimensional system is as fast as establishing the optimal control policy by solving the Riccati equation. For the second system consisting of a high-dimensional gas absorber, IDP was found to be faster than solving the Riccati equation when the number of plates was greater than twenty-five.