Optimal control problems in measure spaces governed by elliptic equations are considered for distributed and Neumann boundary control, which are known to promote sparse solutions. Optimality conditions are derived and some of the structural properties of their solutions, in particular sparsity, are discussed. A framework for their approximation is proposed which is efficient for numerical computations and for which we prove convergence and provide error estimates.