Optimum experimental design (OED) problems are optimization problems in which an experimental setting and decisions on when to measure-the so-called sampling design-are to be determined such that a follow-up parameter estimation yields accurate results for model parameters. In this paper we use the interpretation of OED as optimal control problems with a very particular structure for the analysis of optimal sampling decisions. We introduce the information gain function, motivated by an analysis of necessary conditions of optimality. We highlight differences between problem formulations and propose to use a linear penalization of sampling decisions to overcome the intrinsic ill-conditioning of OED. The results of this paper are independent from the actual numerical method to compute the solution to the OED problem and of the question of local and global optima.