Empirical evidence of an association between internal validity and effect size in randomized controlled trials of low-back pain

M.W. van Tulder, M. Suttorp, S. Morton, L.M. Bouter, P. Shekelle

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

Study Design. We conducted a methodologic study. Objective. The Objective of this study was to assess the validity of the criteria list recommended by the Cochrane Back Review Group Editorial Board by evaluating whether individual items and a total score are associated with effect sizes in randomized controlled trials of back-pain interventions. Summary of Background Data. There is concern that studies of low methodologic quality may exaggerate the effectiveness of treatments for low back pain. We performed this study to examine the association between a common measure of internal validity and the reported magnitude of treatment effects. Methods. We assessed the relationship between the 11 items contained in the Cochrane Back Review Group Internal Validity checklist and effect size in randomized trials of interventions for back pain. Of 267 trials in 15 Cochrane reviews that were eligible for inclusion, 51 were excluded, leaving 216 trials included in the analysis. The scores on the 11 items for each trial were taken from the original review. We extracted effect sizes from each low back pain trial. Results. We found that trials that fulfilled a specific item had smaller effect sizes compared with trials that did not fulfill that item for 10 of the 11 items, and for 6 of the criteria, the absolute difference in effect sizes was 0.10 or greater. The 95% confidence interval of the difference in effect sizes crossed the null value in each case. The number of items fulfilled showed that trials with higher scores consistently reported smaller effect sizes than trials with lower scores. At the thresholds of 5 or 6 items fulfilled, the difference in effect sizes was 0.20 in each case (95% confidence intervals 0.05-0.35 and 0.06-0.34, respectively). Stratified analyses did not support confounding by intervention. Conclusion. We conclude that the 11-item Internal Validity Checklist is associated with effect size in randomized trials of interventions for back pain, and that our data support the use of a sum score of the number of fulfilled items in this list. © 2009, Lippincott Williams & Wilkins.
Original languageEnglish
Pages (from-to)1685-1692
Number of pages8
JournalSpine
Volume34
Issue number16
DOIs
Publication statusPublished - 2009

Fingerprint

Back Pain
Low Back Pain
Randomized Controlled Trials
Checklist
Confidence Intervals
Therapeutics

Cite this

@article{9f26ca96a3c249e78f9b53cc02a751cf,
title = "Empirical evidence of an association between internal validity and effect size in randomized controlled trials of low-back pain",
abstract = "Study Design. We conducted a methodologic study. Objective. The Objective of this study was to assess the validity of the criteria list recommended by the Cochrane Back Review Group Editorial Board by evaluating whether individual items and a total score are associated with effect sizes in randomized controlled trials of back-pain interventions. Summary of Background Data. There is concern that studies of low methodologic quality may exaggerate the effectiveness of treatments for low back pain. We performed this study to examine the association between a common measure of internal validity and the reported magnitude of treatment effects. Methods. We assessed the relationship between the 11 items contained in the Cochrane Back Review Group Internal Validity checklist and effect size in randomized trials of interventions for back pain. Of 267 trials in 15 Cochrane reviews that were eligible for inclusion, 51 were excluded, leaving 216 trials included in the analysis. The scores on the 11 items for each trial were taken from the original review. We extracted effect sizes from each low back pain trial. Results. We found that trials that fulfilled a specific item had smaller effect sizes compared with trials that did not fulfill that item for 10 of the 11 items, and for 6 of the criteria, the absolute difference in effect sizes was 0.10 or greater. The 95{\%} confidence interval of the difference in effect sizes crossed the null value in each case. The number of items fulfilled showed that trials with higher scores consistently reported smaller effect sizes than trials with lower scores. At the thresholds of 5 or 6 items fulfilled, the difference in effect sizes was 0.20 in each case (95{\%} confidence intervals 0.05-0.35 and 0.06-0.34, respectively). Stratified analyses did not support confounding by intervention. Conclusion. We conclude that the 11-item Internal Validity Checklist is associated with effect size in randomized trials of interventions for back pain, and that our data support the use of a sum score of the number of fulfilled items in this list. {\circledC} 2009, Lippincott Williams & Wilkins.",
author = "{van Tulder}, M.W. and M. Suttorp and S. Morton and L.M. Bouter and P. Shekelle",
year = "2009",
doi = "10.1097/BRS.0b013e3181ab6a78",
language = "English",
volume = "34",
pages = "1685--1692",
journal = "Spine",
issn = "0362-2436",
publisher = "Lippincott Williams & Wilkins",
number = "16",

}

Empirical evidence of an association between internal validity and effect size in randomized controlled trials of low-back pain. / van Tulder, M.W.; Suttorp, M.; Morton, S.; Bouter, L.M.; Shekelle, P.

In: Spine, Vol. 34, No. 16, 2009, p. 1685-1692.

Research output: Contribution to JournalArticleAcademicpeer-review

TY - JOUR

T1 - Empirical evidence of an association between internal validity and effect size in randomized controlled trials of low-back pain

AU - van Tulder, M.W.

AU - Suttorp, M.

AU - Morton, S.

AU - Bouter, L.M.

AU - Shekelle, P.

PY - 2009

Y1 - 2009

N2 - Study Design. We conducted a methodologic study. Objective. The Objective of this study was to assess the validity of the criteria list recommended by the Cochrane Back Review Group Editorial Board by evaluating whether individual items and a total score are associated with effect sizes in randomized controlled trials of back-pain interventions. Summary of Background Data. There is concern that studies of low methodologic quality may exaggerate the effectiveness of treatments for low back pain. We performed this study to examine the association between a common measure of internal validity and the reported magnitude of treatment effects. Methods. We assessed the relationship between the 11 items contained in the Cochrane Back Review Group Internal Validity checklist and effect size in randomized trials of interventions for back pain. Of 267 trials in 15 Cochrane reviews that were eligible for inclusion, 51 were excluded, leaving 216 trials included in the analysis. The scores on the 11 items for each trial were taken from the original review. We extracted effect sizes from each low back pain trial. Results. We found that trials that fulfilled a specific item had smaller effect sizes compared with trials that did not fulfill that item for 10 of the 11 items, and for 6 of the criteria, the absolute difference in effect sizes was 0.10 or greater. The 95% confidence interval of the difference in effect sizes crossed the null value in each case. The number of items fulfilled showed that trials with higher scores consistently reported smaller effect sizes than trials with lower scores. At the thresholds of 5 or 6 items fulfilled, the difference in effect sizes was 0.20 in each case (95% confidence intervals 0.05-0.35 and 0.06-0.34, respectively). Stratified analyses did not support confounding by intervention. Conclusion. We conclude that the 11-item Internal Validity Checklist is associated with effect size in randomized trials of interventions for back pain, and that our data support the use of a sum score of the number of fulfilled items in this list. © 2009, Lippincott Williams & Wilkins.

AB - Study Design. We conducted a methodologic study. Objective. The Objective of this study was to assess the validity of the criteria list recommended by the Cochrane Back Review Group Editorial Board by evaluating whether individual items and a total score are associated with effect sizes in randomized controlled trials of back-pain interventions. Summary of Background Data. There is concern that studies of low methodologic quality may exaggerate the effectiveness of treatments for low back pain. We performed this study to examine the association between a common measure of internal validity and the reported magnitude of treatment effects. Methods. We assessed the relationship between the 11 items contained in the Cochrane Back Review Group Internal Validity checklist and effect size in randomized trials of interventions for back pain. Of 267 trials in 15 Cochrane reviews that were eligible for inclusion, 51 were excluded, leaving 216 trials included in the analysis. The scores on the 11 items for each trial were taken from the original review. We extracted effect sizes from each low back pain trial. Results. We found that trials that fulfilled a specific item had smaller effect sizes compared with trials that did not fulfill that item for 10 of the 11 items, and for 6 of the criteria, the absolute difference in effect sizes was 0.10 or greater. The 95% confidence interval of the difference in effect sizes crossed the null value in each case. The number of items fulfilled showed that trials with higher scores consistently reported smaller effect sizes than trials with lower scores. At the thresholds of 5 or 6 items fulfilled, the difference in effect sizes was 0.20 in each case (95% confidence intervals 0.05-0.35 and 0.06-0.34, respectively). Stratified analyses did not support confounding by intervention. Conclusion. We conclude that the 11-item Internal Validity Checklist is associated with effect size in randomized trials of interventions for back pain, and that our data support the use of a sum score of the number of fulfilled items in this list. © 2009, Lippincott Williams & Wilkins.

U2 - 10.1097/BRS.0b013e3181ab6a78

DO - 10.1097/BRS.0b013e3181ab6a78

M3 - Article

VL - 34

SP - 1685

EP - 1692

JO - Spine

JF - Spine

SN - 0362-2436

IS - 16

ER -