IEEE Transactions on Automatic Control, Vol.47, No.6, 979-991, 2002
Burst-level congestion control using hindsight optimization
We consider the burst-level congestion-control problem in a communication network with multiple traffic sources, each modeled as a fully controllable stream of fluid traffic. The controlled traffic shares a common bottleneck node with high-priority cross traffic described by a Markov-modulated fluid (MMF). Each controlled source is assumed to have a unique round-trip delay. The goal is to maximize a linear combination of the throughputs delay, traffic-loss rate, and a fairness metric at the bottleneck node. We introduce a simulation-based congestion-control scheme capable of performing effectively under rapidly varying cross traffic by making use of the provided MMF model of that variation. The control problem is posed as a finite-horizon Markov decision process, and is solved heuristically using a technique called Hindsight Optimization. We provide a detailed derivation of our congestion-control algorithm based on this technique. Our empirical study shows that the control scheme performs significantly better than the conventional proportional-derivative (PD) congestion-control method.
Keywords:communication networks;congestion control;Markov decision processes;Markov-modulated fluid;online simulation;traffic models