Development and validation of a new instrument measuring guideline adherence in clinical practice

Background: Education in evidence-based medicine (EBM) is an important part of the postgraduate training of GPs. Evaluation of its effect on EBM behaviour in daily clinical practice is difficult and instruments are scarce. Working in accordance with guidelines is considered as one of the key indicators of EBM behaviour.

Objective: To develop and validate an instrument assessing guideline adherence of GP trainees in clinical practice.

Methods: We developed an instrument that assesses guideline adherence, taking conscious deviation into account. The instrument assesses guideline adherence on 59 different management decisions (diagnosis N = 17, therapy N = 20, referral N = 22) for 23 conditions as described in 27 different clinical practice guidelines. We validated this instrument using performance data as collected by third-year GP trainees on three important properties: validity, reliability and feasibility.

Results: Performance data were collected by 76 GP trainees on 12106 patient consultations with 12587 different reasons for encounter. Overall, guideline adherence was 82% (95% confidence interval 77-88%). The significant correlation with the national GP knowledge test (r 0.33, P 0.004) showed the instrument to be a valid instrument. Interrater reliabilities (intraclass correlation coefficient) varied between moderate and excellent (0.64-1.00, P < 0.001). The instrument proved feasible with coverage of 24% (N = 3082) of reasons for encounter presented to GP trainees and a mean and median time of 1 minute to score a patient consultation.

Conclusion: This instrument proved valid, reliable and feasible to assess guideline adherence among trainees in the clinical primary care setting.