On the usefulness of interrater reliability coefficients

D. ten Hove; T.D. Jorgensen; L.A. van der Ark

doi:10.1007/978-3-319-77249-3_6

On the usefulness of interrater reliability coefficients

D. ten Hove
, T.D. Jorgensen
, L.A. van der Ark

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

Abstract

For four data sets of different measurement levels, we computed 20 coefficients that estimate interrater reliability. The results show that the coefficients provide very different numerical values when applied to the same data. We discuss possible explanations for the differences among coefficients and suggest further research that is needed to clarify which coefficient a researcher should use to estimate interrater reliability.

Original language	English
Title of host publication	Quantitative Psychology
Subtitle of host publication	The 82nd Annual Meeting of the Psychometric Society, Zurich, Switzerland, 2017
Editors	Jorge Gonzalez, Rianne Janssen, Marie Wiberg, Dylan Molenaar, Steven Culpepper
Publisher	Springer New York LLC
Pages	67-75
Number of pages	9
ISBN (Electronic)	9783319772493
ISBN (Print)	9783319772486, 9783030084103
DOIs	https://doi.org/10.1007/978-3-319-77249-3_6
Publication status	Published - 2018
Externally published	Yes
Event	82nd Annual meeting of the Psychometric Society, 2017 - Zurich, Switzerland Duration: 17 Jul 2017 → 21 Jul 2017

Publication series

Name	Springer Proceedings in Mathematics and Statistics
Volume	233
ISSN (Print)	2194-1009
ISSN (Electronic)	2194-1017

Conference

Conference	82nd Annual meeting of the Psychometric Society, 2017
Country/Territory	Switzerland
City	Zurich
Period	17/07/17 → 21/07/17

Access to Document

10.1007/978-3-319-77249-3_6

Persistent URL (handle)

Persistent link

Cite this

ten Hove, D., Jorgensen, T. D., & van der Ark, L. A. (2018). On the usefulness of interrater reliability coefficients. In J. Gonzalez, R. Janssen, M. Wiberg, D. Molenaar, & S. Culpepper (Eds.), Quantitative Psychology: The 82nd Annual Meeting of the Psychometric Society, Zurich, Switzerland, 2017 (pp. 67-75). (Springer Proceedings in Mathematics and Statistics; Vol. 233). Springer New York LLC. https://doi.org/10.1007/978-3-319-77249-3_6

ten Hove, D. ; Jorgensen, T.D. ; van der Ark, L.A. / On the usefulness of interrater reliability coefficients. Quantitative Psychology: The 82nd Annual Meeting of the Psychometric Society, Zurich, Switzerland, 2017 . editor / Jorge Gonzalez ; Rianne Janssen ; Marie Wiberg ; Dylan Molenaar ; Steven Culpepper. Springer New York LLC, 2018. pp. 67-75 (Springer Proceedings in Mathematics and Statistics).

@inproceedings{73867c2c21b94ca09e5570719c29626c,

title = "On the usefulness of interrater reliability coefficients",

abstract = "For four data sets of different measurement levels, we computed 20 coefficients that estimate interrater reliability. The results show that the coefficients provide very different numerical values when applied to the same data. We discuss possible explanations for the differences among coefficients and suggest further research that is needed to clarify which coefficient a researcher should use to estimate interrater reliability.",

author = "\{ten Hove\}, D. and T.D. Jorgensen and \{van der Ark\}, L.A.",

year = "2018",

doi = "10.1007/978-3-319-77249-3\_6",

language = "English",

isbn = "9783319772486",

series = "Springer Proceedings in Mathematics and Statistics",

publisher = "Springer New York LLC",

pages = "67--75",

editor = "Jorge Gonzalez and Rianne Janssen and Marie Wiberg and Dylan Molenaar and Steven Culpepper",

booktitle = "Quantitative Psychology",

note = "82nd Annual meeting of the Psychometric Society, 2017 ; Conference date: 17-07-2017 Through 21-07-2017",

}

ten Hove, D, Jorgensen, TD & van der Ark, LA 2018, On the usefulness of interrater reliability coefficients. in J Gonzalez, R Janssen, M Wiberg, D Molenaar & S Culpepper (eds), Quantitative Psychology: The 82nd Annual Meeting of the Psychometric Society, Zurich, Switzerland, 2017 . Springer Proceedings in Mathematics and Statistics, vol. 233, Springer New York LLC, pp. 67-75, 82nd Annual meeting of the Psychometric Society, 2017, Zurich, Switzerland, 17/07/17. https://doi.org/10.1007/978-3-319-77249-3_6

On the usefulness of interrater reliability coefficients. / ten Hove, D.; Jorgensen, T.D.; van der Ark, L.A.
Quantitative Psychology: The 82nd Annual Meeting of the Psychometric Society, Zurich, Switzerland, 2017 . ed. / Jorge Gonzalez; Rianne Janssen; Marie Wiberg; Dylan Molenaar; Steven Culpepper. Springer New York LLC, 2018. p. 67-75 (Springer Proceedings in Mathematics and Statistics; Vol. 233).

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - On the usefulness of interrater reliability coefficients

AU - ten Hove, D.

AU - Jorgensen, T.D.

AU - van der Ark, L.A.

PY - 2018

Y1 - 2018

N2 - For four data sets of different measurement levels, we computed 20 coefficients that estimate interrater reliability. The results show that the coefficients provide very different numerical values when applied to the same data. We discuss possible explanations for the differences among coefficients and suggest further research that is needed to clarify which coefficient a researcher should use to estimate interrater reliability.

AB - For four data sets of different measurement levels, we computed 20 coefficients that estimate interrater reliability. The results show that the coefficients provide very different numerical values when applied to the same data. We discuss possible explanations for the differences among coefficients and suggest further research that is needed to clarify which coefficient a researcher should use to estimate interrater reliability.

UR - https://www.scopus.com/pages/publications/85045973388

UR - https://www.scopus.com/pages/publications/85045973388#tab=citedBy

U2 - 10.1007/978-3-319-77249-3_6

DO - 10.1007/978-3-319-77249-3_6

M3 - Conference contribution

SN - 9783319772486

SN - 9783030084103

T3 - Springer Proceedings in Mathematics and Statistics

SP - 67

EP - 75

BT - Quantitative Psychology

A2 - Gonzalez, Jorge

A2 - Janssen, Rianne

A2 - Wiberg, Marie

A2 - Molenaar, Dylan

A2 - Culpepper, Steven

PB - Springer New York LLC

T2 - 82nd Annual meeting of the Psychometric Society, 2017

Y2 - 17 July 2017 through 21 July 2017

ER -

ten Hove D, Jorgensen TD, van der Ark LA. On the usefulness of interrater reliability coefficients. In Gonzalez J, Janssen R, Wiberg M, Molenaar D, Culpepper S, editors, Quantitative Psychology: The 82nd Annual Meeting of the Psychometric Society, Zurich, Switzerland, 2017 . Springer New York LLC. 2018. p. 67-75. (Springer Proceedings in Mathematics and Statistics). Epub 2018 Apr 21. doi: 10.1007/978-3-319-77249-3_6

On the usefulness of interrater reliability coefficients

Abstract

Publication series

Conference

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this