Abstract
OBJECTIVE: To determine the reproducibility of biomedical systematic review search strategies.
STUDY DESIGN AND SETTING: A cross-sectional reproducibility study was conducted on a random sample of 100 systematic reviews indexed in MEDLINE in November 2021. The primary outcome measure is the percentage of systematic reviews for which all database searches can be reproduced, operationalized as fulfilling 6 key PRISMA-S reporting guideline items and having all database searches reproduced within 10% of the number of original results. Key reporting guideline items included database name, multi-database searching, full search strategies, limits and restrictions, date(s) of searches, and total records.
RESULTS: The 100 systematic review articles contained 453 database searches. Only 22 (4.9%) database searches reported all six PRISMA-S items. Forty-seven (10.4%) database searches could be reproduced within 10% of the number of results from the original search; 6 searches differed by more than 1000% between the originally reported number of results and the reproduction. Only one systematic review article provided the necessary search details to be fully reproducible.
CONCLUSION: Systematic review search reporting is poor. To correct this will require a multi-faceted response from authors, peer reviewers, journal editors, and database providers.
Original language | English |
---|---|
Article number | 111229 |
Pages (from-to) | 1-12 |
Number of pages | 12 |
Journal | Journal of clinical epidemiology |
Volume | 166 |
Early online date | 3 Dec 2023 |
DOIs | |
Publication status | Published - Feb 2024 |
Bibliographical note
Copyright © 2023 The Author(s). Published by Elsevier Inc. All rights reserved.Funding
No specific funding was received for this study. It is part of M.L.R.’s self-funded PhD project registered at Maastricht University, the Netherlands, in collaboration with the BMJ, United Kingdom. M.L.R. and D.M. are co-authors of PRISMA-S, which is used in this study. T.J.B., C.P., L.M.B., and M.P.Z. have no competing interests. S.S. is a full-time employee of BMJ. J.J.K. is a statistical reviewer for BMJ. D.M. is an editorial board member of Journal of Clinical Epidemiology.
Funders | Funder number |
---|---|
BMJ |