Biomarkers offer significant potential for diagnosis and treatment of complex disorders such as asthma, epilepsy, autism, Parkinson's, and Alzheimer's, as well as many others. In many cases, however, there is little consensus on what an appropriate biomarker would be. Consequently, biomarker identification is an important area of research for which a link between physiological measurements and the presence/absence or severity of a disorder can be established. This is nontrivial due to both the curse of dimensionality and because the number of measurements per trial often exceeds the number of trial participants. Overfitting of potential biomarkers is thus a significant problem that needs to be addressed. This paper highlights similarities between the biomarker identification problem and the parameter estimation problem, more specifically the regularization used for avoiding overfitting. Parallels between the underlying methodologies are pointed out and opportunities for advancing the systems' concepts are discussed. Finally, a candidate biomarker for diagnosis of autism spectrum disorder is identified from a data set comprising metabolic measurements from four separate clinical trials to illustrate the procedure outlined in this work.