Background: The Four-Dimensional Symptom Questionnaire (4DSQ) is a self-report questionnaire that has been developed in primary care to distinguish non-specific general distress from depression, anxiety and somatization. The purpose of this paper is to evaluate its criterion and construct validity. Methods: Data from 10 different primary care studies have been used. Criterion validity was assessed by comparing the 4DSQ scores with clinical diagnoses, the GPs' diagnosis of any psychosocial problem for Distress, standardised psychiatric diagnoses for Depression and Anxiety, and GPs' suspicion of somatization for Somatization. ROC analyses and logistic regression analyses were used to examine the associations. Construct validity was evaluated by investigating the inter-correlations between the scales, the factorial structure, the associations with other symptom questionnaires, and the associations with stress, personality and social functioning. The factorial structure of the 4DSQ was assessed through confirmatory factor analysis (CFA). The associations with other questionnaires were assessed with Pearson correlations and regression analyses. Results: Regarding criterion validity, the Distress scale was associated with any psychosocial diagnosis (area under the ROC curve [AUC] 0.79), the Depression scale was associated with major depression (AUC = 0.83), the Anxiety scale was associated with anxiety disorder (AUC = 0.66), and the Somatization scale was associated with the GPs' suspicion of somatization (AUC = 0.65). Regarding the construct validity, the 4DSQ scales appeared to have considerable intercorrelations (r = 0.35-0.71). However, 30-40% of the variance of each scale was unique for that scale. CFA confirmed the 4-factor structure with a comparative fit index (CFI) of 0.92. The 4DSQ scales correlated with most other questionnaires measuring corresponding constructs. However, the 4DSQ Distress scale appeared to correlate with some other depression scales more than the 4DSQ Depression scale. Measures of stress (i.e. life events, psychosocial problems, and work stress) were mainly associated with Distress, while Distress, in turn, was mainly associated with psychosocial dysfunctioning, including sick leave. Conclusion: The 4DSQ seems to be a valid self-report questionnaire to measure distress, depression, anxiety and somatization in primary care patients. The 4DSQ Distress scale appears to measure the most general, most common, expression of psychological problems. © 2006 Terluin et al; licensee BioMed Central Ltd.