A novel optimization algorithmic framework based on dynamic programming is proposed for solving multi-product supply chain management problems with manufacturing and distribution decisions under demand uncertainty. To generate reliable suboptimal policy for simulation and restricted state space identification, a deterministic mathematical programming (MILP) approach is utilized. The simulation data with fixed action profiles obtained from the MILPs with different demand patterns is directly utilized for real-time decision making with initial 'profit-to-go' values.