Regular exercise has often been suggested to have beneficial effects on cognition, but empirical findings are mixed because of heterogeneity in sample composition (age and sex); the cognitive domain being investigated; the definition and reliability of exercise behavior measures; and study design (e.g., observational versus experimental). Our aim was to scrutinize the domain specificity of exercise effects on cognition, while controlling for the other sources of heterogeneity. In a population based sample consisting of 472 males and 668 females (aged 10-86. years old) we administered the Computerized Neurocognitive Battery (CNB), which provided accuracy and speed measures of abstraction and mental flexibility, attention, working memory, memory (verbal, face, and spatial), language and nonverbal reasoning, spatial ability, emotion identification, emotion- and age differentiation, sensorimotor speed, and motor speed. Using univariate and multivariate regression models, CNB scores were associated with participants' average energy expenditure per week (weekly METhours), which were derived from a questionnaire on voluntary regular leisure time exercise behavior.Univariate models yielded generally positive associations between weekly METhours and cognitive accuracy and speed, but multivariate modeling demonstrated that direct relations were small and centered around zero. The largest and only significant effect size (β = 0.11, p < 0.001) was on the continuous performance test, which measures attention. Our results suggest that in the base population, any chronic effects of voluntary regular leisure time exercise on cognition are limited. Only a relation between exercise and attention inspires confidence.