A solution to dependency: using multilevel analysis to accommodate nested data

E. Aarts, M. Verhage, J.V. Veenvliet, C.V. Dolan, S. van der Sluis

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

In neuroscience, experimental designs in which multiple observations are collected from a single research object (for example, multiple neurons from one animal) are common: 53% of 314 reviewed papers from five renowned journals included this type of data. These so-called 'nested designs' yield data that cannot be considered to be independent, and so violate the independency assumption of conventional statistical methods such as the t test. Ignoring this dependency results in a probability of incorrectly concluding that an effect is statistically significant that is far higher (up to 80%) than the nominal level (usually set at 5%). We discuss the factors affecting the type I error rate and the statistical power in nested data, methods that accommodate dependency between observations and ways to determine the optimal study design when data are nested. Notably, optimization of experimental designs nearly always concerns collection of more truly independent observations, rather than more observations from one research object. © 2014 Nature America, Inc. All rights reserved.
Original languageEnglish
Pages (from-to)491-496
Number of pages6
JournalNature Neuroscience
Volume17
Issue number4
DOIs
Publication statusPublished - 2014

Fingerprint

Multilevel Analysis
Research Design
Neurosciences
Research
Neurons

Cite this

@article{6b2fa96f81ed4e6d98a3d2fcf58ca1ee,
title = "A solution to dependency: using multilevel analysis to accommodate nested data",
abstract = "In neuroscience, experimental designs in which multiple observations are collected from a single research object (for example, multiple neurons from one animal) are common: 53{\%} of 314 reviewed papers from five renowned journals included this type of data. These so-called 'nested designs' yield data that cannot be considered to be independent, and so violate the independency assumption of conventional statistical methods such as the t test. Ignoring this dependency results in a probability of incorrectly concluding that an effect is statistically significant that is far higher (up to 80{\%}) than the nominal level (usually set at 5{\%}). We discuss the factors affecting the type I error rate and the statistical power in nested data, methods that accommodate dependency between observations and ways to determine the optimal study design when data are nested. Notably, optimization of experimental designs nearly always concerns collection of more truly independent observations, rather than more observations from one research object. {\circledC} 2014 Nature America, Inc. All rights reserved.",
author = "E. Aarts and M. Verhage and J.V. Veenvliet and C.V. Dolan and {van der Sluis}, S.",
year = "2014",
doi = "10.1038/nn.3648",
language = "English",
volume = "17",
pages = "491--496",
journal = "Nature Neuroscience",
issn = "1097-6256",
publisher = "Nature Publishing Group",
number = "4",

}

A solution to dependency: using multilevel analysis to accommodate nested data. / Aarts, E.; Verhage, M.; Veenvliet, J.V.; Dolan, C.V.; van der Sluis, S.

In: Nature Neuroscience, Vol. 17, No. 4, 2014, p. 491-496.

Research output: Contribution to JournalArticleAcademicpeer-review

TY - JOUR

T1 - A solution to dependency: using multilevel analysis to accommodate nested data

AU - Aarts, E.

AU - Verhage, M.

AU - Veenvliet, J.V.

AU - Dolan, C.V.

AU - van der Sluis, S.

PY - 2014

Y1 - 2014

N2 - In neuroscience, experimental designs in which multiple observations are collected from a single research object (for example, multiple neurons from one animal) are common: 53% of 314 reviewed papers from five renowned journals included this type of data. These so-called 'nested designs' yield data that cannot be considered to be independent, and so violate the independency assumption of conventional statistical methods such as the t test. Ignoring this dependency results in a probability of incorrectly concluding that an effect is statistically significant that is far higher (up to 80%) than the nominal level (usually set at 5%). We discuss the factors affecting the type I error rate and the statistical power in nested data, methods that accommodate dependency between observations and ways to determine the optimal study design when data are nested. Notably, optimization of experimental designs nearly always concerns collection of more truly independent observations, rather than more observations from one research object. © 2014 Nature America, Inc. All rights reserved.

AB - In neuroscience, experimental designs in which multiple observations are collected from a single research object (for example, multiple neurons from one animal) are common: 53% of 314 reviewed papers from five renowned journals included this type of data. These so-called 'nested designs' yield data that cannot be considered to be independent, and so violate the independency assumption of conventional statistical methods such as the t test. Ignoring this dependency results in a probability of incorrectly concluding that an effect is statistically significant that is far higher (up to 80%) than the nominal level (usually set at 5%). We discuss the factors affecting the type I error rate and the statistical power in nested data, methods that accommodate dependency between observations and ways to determine the optimal study design when data are nested. Notably, optimization of experimental designs nearly always concerns collection of more truly independent observations, rather than more observations from one research object. © 2014 Nature America, Inc. All rights reserved.

U2 - 10.1038/nn.3648

DO - 10.1038/nn.3648

M3 - Article

VL - 17

SP - 491

EP - 496

JO - Nature Neuroscience

JF - Nature Neuroscience

SN - 1097-6256

IS - 4

ER -