We study several aspects of covariance control problems over martingale processes in Rd with constraints on the terminal distribution, arising from the theory of repeated games with incomplete information. We show that these control problems are the limits of discrete-time stochastic optimization problems called problems of maximal variation of martingales, meaning that sequences of optimizers for problems of length n, seen as piecewise constant processes on the uniform partition of [0, 1], define relatively compact sequences having all their limit points in the set of optimizers of the control problem. Optimal solutions of this limit problem are then characterized using convex duality techniques, and the dual problem is shown to be an unconstrained stochastic control problem characterized by a second order nonlinear PDE of HJB type. We deduce from this dual relationship that solutions of the control problem are the images by the spatial gradient of the solution of the HJB equation of the solutions of the dual stochastic control problem using tools from optimal transport theory.