Do We Overestimate the Within-Variability? The Impact of Measurement Error on Intraclass Coefficient Estimation

Rafael Wilms*, Ralf Lanwehr, Andreas Kastenmüller

*Corresponding author for this work

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

Many psychological phenomena have a multilevel structure (e.g., individuals within teams or events within individuals). In these cases, the proportion of between-variance to total-variance (i.e., the sum between-variance and within-variance) is of special importance and usually estimated by the intraclass coefficient (1)1 [ICC(1)]. Our contribution firstly shows via mathematical proof that measurement error increases the within-variance, which in turn decreases the ICC(1). Further, we provide a numerical example, and examine the RMSEs, alpha error rates and the inclusion of zero in the confidence intervals for ICC(1) estimation with and without measurement error. Secondly, we propose two corrections [i.e., the reliability-adjusted ICC(1) and the measurement model-based ICC(1)] that yield correct estimates for the ICC(1), and prove that they are unaffected by measurement error mathematically. Finally, we discuss our findings, point out examples of the underestimation of the ICC(1) in the literature, and reinterpret the results of these examples in the light of our new estimator. We also illustrate the potential application of our work to other ICCs. Finally, we conclude that measurement error distorts the ICC(1) to a non-negligible extent.

Original languageEnglish
Article number825
JournalFrontiers in Psychology
Volume11
DOIs
Publication statusPublished - 19 May 2020
Externally publishedYes

Bibliographical note

Publisher Copyright:
© Copyright © 2020 Wilms, Lanwehr and Kastenmüller.

Keywords

  • between-variance
  • intraclass coefficient
  • measurement error
  • reliability
  • within-variance

Fingerprint

Dive into the research topics of 'Do We Overestimate the Within-Variability? The Impact of Measurement Error on Intraclass Coefficient Estimation'. Together they form a unique fingerprint.

Cite this