Noncompliant responding: Comparing exclusion criteria in MTurk personality research to improve data quality

Ard J. Barends*, Reinout E. de Vries

*Corresponding author for this work

Research output: Contribution to JournalArticleAcademicpeer-review

152 Downloads (Pure)

Abstract

Studies on Amazon Mechanical Turk (MTurk) often include check questions in personality inventories to ensure data quality. However, a subset of MTurk workers may give only meaningful responses to these checks while giving noncompliant responses to the other questions. We demonstrate in an analysis of five MTurk datasets using the statistical approach of Lee and Ashton (2018) that this selectively responsive subset can be detected on the HEXACO personality inventory. Our lower bound estimate is that at least 2% in each sample did not get caught with the check questions while giving noncompliant responses on the personality inventory. Overall, researchers who strive to remove noise due to noncompliant responding may benefit from complementing check questions with a statistical approach.

Original languageEnglish
Pages (from-to)84-89
Number of pages6
JournalPersonality and Individual Differences
Volume143
Early online date21 Feb 2019
DOIs
Publication statusPublished - 1 Jun 2019

Keywords

  • HEXACO
  • MTurk
  • Noncompliant responding
  • Online research

Fingerprint

Dive into the research topics of 'Noncompliant responding: Comparing exclusion criteria in MTurk personality research to improve data quality'. Together they form a unique fingerprint.

Cite this