The strong focus on positive results in abstracts may cause bias in systematic reviews: a case study on abstract reporting bias

Bram Duyx*, Gerard M.H. Swaen, Miriam J.E. Urlings, Lex M. Bouter, Maurice P. Zeegers

*Corresponding author for this work

Research output: Contribution to JournalReview articleAcademicpeer-review

Abstract

BACKGROUND: Research articles tend to focus on positive findings in their abstract, especially if multiple outcomes have been studied. At the same time, search queries in databases are generally limited to the abstract, title and keywords fields of an article. Negative findings are therefore less likely to be detected by systematic searches and to appear in systematic reviews. We aim to assess the occurrence of this 'abstract reporting bias' and quantify its impact in the literature on the association between diesel exhaust exposure (DEE) and bladder cancer.

METHODS: We set up a broad search query related to DEE and cancer in general. Full-texts of the articles identified in the search output were manually scanned. Articles were included if they reported, anywhere in the full-text, the association between DEE and bladder cancer. We assume that the use of a broad search query and manual full-text scanning allowed us to catch all the relevant articles, including those in which bladder cancer was not mentioned in the abstract, title or keywords.

RESULTS: We identified 28 articles. Only 12 of these (43%) had mentioned bladder in their abstract, title or keywords. A meta-analysis based on these 12 detectable articles yielded a pooled risk estimate of 1.10 (95% confidence interval [CI] 0.97-1.25), whereas the meta-analysis based on all 28 articles yielded a pooled estimate of 1.03 (95% CI 0.96-1.11).

CONCLUSIONS: This case study on abstract reporting bias shows that (a) more than half of all relevant articles were missed by a conventional search query and (b) this led to an overestimation of the pooled effect. Detection of articles will be improved if all studied exposure and outcome variables are reported in the keywords. The restriction on the maximum number of keywords should be lifted.

Original languageEnglish
Article number174
Pages (from-to)1-8
Number of pages8
JournalSystematic Reviews
Volume8
Issue number1
DOIs
Publication statusPublished - 17 Jul 2019

Keywords

  • Bladder cancer
  • Diesel exhaust exposure
  • Epidemiology
  • Reporting bias
  • Search engines
  • Systematic reviews

Fingerprint Dive into the research topics of 'The strong focus on positive results in abstracts may cause bias in systematic reviews: a case study on abstract reporting bias'. Together they form a unique fingerprint.

  • Cite this