Abstract
OBJECTIVE: To evaluate the impact of adding librarians and information specialists (LIS) as methodological peer reviewers to the formal journal peer review process on the quality of search reporting and risk of bias in systematic review searches in the medical literature.
DESIGN: Pragmatic two-group parallel randomised controlled trial.
SETTING: Three biomedical journals.
PARTICIPANTS: Systematic reviews and related evidence synthesis manuscripts submitted to The BMJ, BMJ Open and BMJ Medicine and sent out for peer review from 3 January 2023 to 1 September 2023. Randomisation (allocation ratio, 1:1) was stratified by journal and used permuted blocks (block size=4). Of 2670 manuscripts sent to peer review during study enrollment, 400 met inclusion criteria and were randomised (62 The BMJ, 334 BMJ Open, 4 BMJ Medicine). 76 manuscripts were revised and resubmitted in the intervention group and 90 in the control group by 2 January 2024.
INTERVENTIONS: All manuscripts followed usual journal practice for peer review, but those in the intervention group had an additional (LIS) peer reviewer invited.
MAIN OUTCOME MEASURES: The primary outcomes are the differences in first revision manuscripts between intervention and control groups in the quality of reporting and risk of bias. Quality of reporting was measured using four prespecified PRISMA-S items. Risk of bias was measured using ROBIS Domain 2. Assessments were done in duplicate and assessors were blinded to group allocation. Secondary outcomes included differences between groups for each individual PRISMA-S and ROBIS Domain 2 item. The difference in the proportion of manuscripts rejected as the first decision post-peer review between the intervention and control groups was an additional outcome.
RESULTS: Differences in the proportion of adequately reported searches (4.4% difference, 95% CI: -2.0% to 10.7%) and risk of bias in searches (0.5% difference, 95% CI: -13.7% to 14.6%) showed no statistically significant differences between groups. By 4 months post-study, 98 intervention and 70 control group manuscripts had been rejected after peer review (13.8% difference, 95% CI: 3.9% to 23.8%).
CONCLUSIONS: Inviting LIS peer reviewers did not impact adequate reporting or risk of bias of searches in first revision manuscripts of biomedical systematic reviews and related review types, though LIS peer reviewers may have contributed to a higher rate of rejection after peer review.
TRIAL REGISTRATION NUMBER: Open Science Framework: https://doi.org/10.17605/OSF.IO/W4CK2.
| Original language | English |
|---|---|
| Article number | bmjebm-2024-113527 |
| Journal | BMJ evidence-based medicine |
| DOIs | |
| Publication status | E-pub ahead of print - 11 Mar 2025 |
Bibliographical note
© Author(s) (or their employer(s)) 2025. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ Group.Funding
All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/disclosure-of-interest/ and declare: no financial support for this specific study; MLR is the lead author of PRISMA-S and has received funding from the Medical Library Association and Baltic American Freedom Foundation to teach PRISMA-S; DM is a member of the BMJ North American advisory committee; SS is a full-time employee of BMJ Publishing Group, regularly undertakes research on the publication process but is not involved in decision-making on manuscripts; KN, HKGN, and MLR are managers of the Librarian Peer Reviewer Database; this research was conducted under a confidentiality agreement and is part of an ongoing PhD collaboration between BMJ Publishing Group and the team Meta-Research at Maastricht University (UM) on the responsible conduct of publishing scientific research; BMJ Publishing Group Ltd \u201CBMJ \u201D is a private limited company incorporated in England and Wales, registration number 3102371; UM is a public legal entity in the Netherlands, registration number 50169181; this is part of MLR\u2019s self-funded BMJ/UM PhD program through Maastricht University. Authors express their own opinions and not necessarily of their funders or employers.
| Funders | Funder number |
|---|---|
| Universiteit Maastricht | |
| Medical Library Association and Baltic American Freedom Foundation | |
| BMJ Publishing Group | |
| team Meta-Research at Maastricht University | |
| University of Montana | 3102371, 50169181 |