When asked to perform the same task, different individuals exhibit markedly different patterns of brain activity. This variability is often attributed to volatile factors, such as task strategy or compliance. We propose that individual differences in brain responses are, to a large degree, inherent to the brain and can be predicted from task-independent measurements collected at rest. Using a large set of task conditions, spanning several behavioral domains, we train a simple model that relates task-independent measurements to task activity and evaluate the model by predicting task activation maps for unseen subjects using magnetic resonance imaging. Our model can accurately predict individual differences in brain activity and highlights a coupling between brain connectivity and function that can be captured at the level of individual subjects.