We use the Markov chain approximation method to construct approximations for the solution of the mean field game (MFG) with reflecting barriers studied in [E. Bayraktar, A. Budhiraja, and A. Cohen, Ann. Appl. Probab., to appear]. The MFG is formulated in terms of a controlled reflected diffusion with a cost function that depends on the reflection terms in addition to the standard variables: state, control, and the mean field term. This MFG arises from the asymptotic analysis of an N-player game for single server queues with strategic servers. By showing that our scheme is an almost contraction, we establish the convergence of this numerical scheme over a small time interval.