Reproducibility and replicability of rodent phenotyping in preclinical studies

Neri Kafkafi; Joseph Agassi; Elissa J. Chesler; John C. Crabbe; Wim E. Crusio; David Eilam; Robert Gerlai; Ilan Golani; Alex Gomez-Marin; Ruth Heller; Fuad Iraqi; Iman Jaljuli; Natasha A. Karp; Hugh Morgan; George Nicholson; Donald W. Pfaff; S. Helene Richter; Philip B. Stark; Oliver Stiedl; Victoria Stodden; Lisa M. Tarantino; Valter Tucci; William Valdar; Robert W. Williams; Hanno Würbel; Yoav Benjamini

doi:10.1016/j.neubiorev.2018.01.003

Reproducibility and replicability of rodent phenotyping in preclinical studies

Neri Kafkafi^*, Joseph Agassi, Elissa J. Chesler, John C. Crabbe, Wim E. Crusio, David Eilam, Robert Gerlai, Ilan Golani, Alex Gomez-Marin, Ruth Heller, Fuad Iraqi, Iman Jaljuli, Natasha A. Karp, Hugh Morgan, George Nicholson, Donald W. Pfaff, S. Helene Richter, Philip B. Stark, Oliver Stiedl, Victoria StoddenLisa M. Tarantino, Valter Tucci, William Valdar, Robert W. Williams, Hanno Würbel, Yoav Benjamini

^*Corresponding author for this work

Research output: Contribution to Journal › Review article › Academic › peer-review

Abstract

The scientific community is increasingly concerned with the proportion of published “discoveries” that are not replicated in subsequent studies. The field of rodent behavioral phenotyping was one of the first to raise this concern, and to relate it to other methodological issues: the complex interaction between genotype and environment; the definitions of behavioral constructs; and the use of laboratory mice and rats as model species for investigating human health and disease mechanisms. In January 2015, researchers from various disciplines gathered at Tel Aviv University to discuss these issues. The general consensus was that the issue is prevalent and of concern, and should be addressed at the statistical, methodological and policy levels, but is not so severe as to call into question the validity and the usefulness of model organisms as a whole. Well-organized community efforts, coupled with improved data and metadata sharing, have a key role in identifying specific problems and promoting effective solutions. Replicability is closely related to validity, may affect generalizability and translation of findings, and has important ethical implications.

Original language	English
Pages (from-to)	218-232
Number of pages	15
Journal	Neuroscience and Biobehavioral Reviews
Volume	87
Early online date	31 Jan 2018
DOIs	https://doi.org/10.1016/j.neubiorev.2018.01.003
Publication status	Published - Apr 2018

Funding

This work is supported by European Research Council grants FP7/2007-2013 ERC agreement no [294519]-PSARPS (YB, NK, IG, IJ), by NIH AA018776 (EJC), by NIH DA045401 (YB, NK) and by the Spanish Ministry of Economy and Competitivity (Severo Ochoa Center of Excellence programme SEV2013-0317 start-up funds to AGM). We thank Prof. Allan Kalueff for many useful comments. This paper is the result of a scientific meeting in Tel Aviv University. Videos of the full lectures can be accessed at the following links.

Funders	Funder number
Ministerio de Economía y Competitividad
National Institutes of Health
European Research Council
National Institute on Alcohol Abuse and Alcoholism	P60AA010760, U01AA013519, R01AA018776
European Commission	322576
National Institute on Drug Abuse	R01DA045401
Seventh Framework Programme	294519
National Institute of General Medical Sciences	R01GM104125

Keywords

Data sharing
False discoveries
GxE interaction
Heterogenization
Replicability
Reproducibility
Validity

Access to Document

10.1016/j.neubiorev.2018.01.003

Persistent URL (handle)

Persistent link

Cite this

Kafkafi, N., Agassi, J., Chesler, E. J., Crabbe, J. C., Crusio, W. E., Eilam, D., Gerlai, R., Golani, I., Gomez-Marin, A., Heller, R., Iraqi, F., Jaljuli, I., Karp, N. A., Morgan, H., Nicholson, G., Pfaff, D. W., Richter, S. H., Stark, P. B., Stiedl, O., ... Benjamini, Y. (2018). Reproducibility and replicability of rodent phenotyping in preclinical studies. Neuroscience and Biobehavioral Reviews, 87, 218-232. https://doi.org/10.1016/j.neubiorev.2018.01.003

@article{0ba0311fe29148ec85b842bf57877ff2,

title = "Reproducibility and replicability of rodent phenotyping in preclinical studies",

abstract = "The scientific community is increasingly concerned with the proportion of published “discoveries” that are not replicated in subsequent studies. The field of rodent behavioral phenotyping was one of the first to raise this concern, and to relate it to other methodological issues: the complex interaction between genotype and environment; the definitions of behavioral constructs; and the use of laboratory mice and rats as model species for investigating human health and disease mechanisms. In January 2015, researchers from various disciplines gathered at Tel Aviv University to discuss these issues. The general consensus was that the issue is prevalent and of concern, and should be addressed at the statistical, methodological and policy levels, but is not so severe as to call into question the validity and the usefulness of model organisms as a whole. Well-organized community efforts, coupled with improved data and metadata sharing, have a key role in identifying specific problems and promoting effective solutions. Replicability is closely related to validity, may affect generalizability and translation of findings, and has important ethical implications.",

keywords = "Data sharing, False discoveries, GxE interaction, Heterogenization, Replicability, Reproducibility, Validity",

author = "Neri Kafkafi and Joseph Agassi and Chesler, \{Elissa J.\} and Crabbe, \{John C.\} and Crusio, \{Wim E.\} and David Eilam and Robert Gerlai and Ilan Golani and Alex Gomez-Marin and Ruth Heller and Fuad Iraqi and Iman Jaljuli and Karp, \{Natasha A.\} and Hugh Morgan and George Nicholson and Pfaff, \{Donald W.\} and Richter, \{S. Helene\} and Stark, \{Philip B.\} and Oliver Stiedl and Victoria Stodden and Tarantino, \{Lisa M.\} and Valter Tucci and William Valdar and Williams, \{Robert W.\} and Hanno W{\"u}rbel and Yoav Benjamini",

year = "2018",

month = apr,

doi = "10.1016/j.neubiorev.2018.01.003",

language = "English",

volume = "87",

pages = "218--232",

journal = "Neuroscience and Biobehavioral Reviews",

issn = "0149-7634",

publisher = "Elsevier Ltd",

}

Kafkafi, N, Agassi, J, Chesler, EJ, Crabbe, JC, Crusio, WE, Eilam, D, Gerlai, R, Golani, I, Gomez-Marin, A, Heller, R, Iraqi, F, Jaljuli, I, Karp, NA, Morgan, H, Nicholson, G, Pfaff, DW, Richter, SH, Stark, PB, Stiedl, O, Stodden, V, Tarantino, LM, Tucci, V, Valdar, W, Williams, RW, Würbel, H & Benjamini, Y 2018, 'Reproducibility and replicability of rodent phenotyping in preclinical studies', Neuroscience and Biobehavioral Reviews, vol. 87, pp. 218-232. https://doi.org/10.1016/j.neubiorev.2018.01.003

TY - JOUR

T1 - Reproducibility and replicability of rodent phenotyping in preclinical studies

AU - Kafkafi, Neri

AU - Agassi, Joseph

AU - Chesler, Elissa J.

AU - Crabbe, John C.

AU - Crusio, Wim E.

AU - Eilam, David

AU - Gerlai, Robert

AU - Golani, Ilan

AU - Gomez-Marin, Alex

AU - Heller, Ruth

AU - Iraqi, Fuad

AU - Jaljuli, Iman

AU - Karp, Natasha A.

AU - Morgan, Hugh

AU - Nicholson, George

AU - Pfaff, Donald W.

AU - Richter, S. Helene

AU - Stark, Philip B.

AU - Stiedl, Oliver

AU - Stodden, Victoria

AU - Tarantino, Lisa M.

AU - Tucci, Valter

AU - Valdar, William

AU - Williams, Robert W.

AU - Würbel, Hanno

AU - Benjamini, Yoav

PY - 2018/4

Y1 - 2018/4

N2 - The scientific community is increasingly concerned with the proportion of published “discoveries” that are not replicated in subsequent studies. The field of rodent behavioral phenotyping was one of the first to raise this concern, and to relate it to other methodological issues: the complex interaction between genotype and environment; the definitions of behavioral constructs; and the use of laboratory mice and rats as model species for investigating human health and disease mechanisms. In January 2015, researchers from various disciplines gathered at Tel Aviv University to discuss these issues. The general consensus was that the issue is prevalent and of concern, and should be addressed at the statistical, methodological and policy levels, but is not so severe as to call into question the validity and the usefulness of model organisms as a whole. Well-organized community efforts, coupled with improved data and metadata sharing, have a key role in identifying specific problems and promoting effective solutions. Replicability is closely related to validity, may affect generalizability and translation of findings, and has important ethical implications.

AB - The scientific community is increasingly concerned with the proportion of published “discoveries” that are not replicated in subsequent studies. The field of rodent behavioral phenotyping was one of the first to raise this concern, and to relate it to other methodological issues: the complex interaction between genotype and environment; the definitions of behavioral constructs; and the use of laboratory mice and rats as model species for investigating human health and disease mechanisms. In January 2015, researchers from various disciplines gathered at Tel Aviv University to discuss these issues. The general consensus was that the issue is prevalent and of concern, and should be addressed at the statistical, methodological and policy levels, but is not so severe as to call into question the validity and the usefulness of model organisms as a whole. Well-organized community efforts, coupled with improved data and metadata sharing, have a key role in identifying specific problems and promoting effective solutions. Replicability is closely related to validity, may affect generalizability and translation of findings, and has important ethical implications.

KW - Data sharing

KW - False discoveries

KW - GxE interaction

KW - Heterogenization

KW - Replicability

KW - Reproducibility

KW - Validity

UR - https://www.scopus.com/pages/publications/85042385750

UR - https://www.scopus.com/inward/citedby.url?scp=85042385750&partnerID=8YFLogxK

U2 - 10.1016/j.neubiorev.2018.01.003

DO - 10.1016/j.neubiorev.2018.01.003

M3 - Review article

AN - SCOPUS:85042385750

SN - 0149-7634

VL - 87

SP - 218

EP - 232

JO - Neuroscience and Biobehavioral Reviews

JF - Neuroscience and Biobehavioral Reviews

ER -

Reproducibility and replicability of rodent phenotyping in preclinical studies

Abstract

Funding

Keywords

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this