The Wisdom of the Inner Crowd in Three Large Natural Experiments

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

The quality of decisions depends on the accuracy of estimates of relevant quantities. According to the wisdom of crowds principle, accurate estimates can be obtained by combining the judgements of different individuals 1,2. This principle has been successfully applied to improve, for example, economic forecasts 3-5, medical judgements 6-9 and meteorological predictions 10-13. Unfortunately, there are many situations in which it is infeasible to collect judgements of others. Recent research proposes that a similar principle applies to repeated judgements from the same person 14. This paper tests this promising approach on a large scale in a real-world context. Using proprietary data comprising 1.2 million observations from three incentivized guessing competitions, we find that within-person aggregation indeed improves accuracy and that the method works better when there is a time delay between subsequent judgements. However, the benefit pales against that of between-person aggregation: the average of a large number of judgements from the same person is barely better than the average of two judgements from different people.

LanguageEnglish
Pages21-26
JournalNature Human Behaviour
Volume2
Issue number1
Early online date11 Dec 2017
DOIs
StatePublished - Jan 2018

Fingerprint

Economics
Research

Cite this

@article{e9dc35642c084de78a3ae8e74a8d9fac,
title = "The Wisdom of the Inner Crowd in Three Large Natural Experiments",
abstract = "The quality of decisions depends on the accuracy of estimates of relevant quantities. According to the wisdom of crowds principle, accurate estimates can be obtained by combining the judgements of different individuals 1,2. This principle has been successfully applied to improve, for example, economic forecasts 3-5, medical judgements 6-9 and meteorological predictions 10-13. Unfortunately, there are many situations in which it is infeasible to collect judgements of others. Recent research proposes that a similar principle applies to repeated judgements from the same person 14. This paper tests this promising approach on a large scale in a real-world context. Using proprietary data comprising 1.2 million observations from three incentivized guessing competitions, we find that within-person aggregation indeed improves accuracy and that the method works better when there is a time delay between subsequent judgements. However, the benefit pales against that of between-person aggregation: the average of a large number of judgements from the same person is barely better than the average of two judgements from different people.",
author = "{van Dolder}, Dennie and {van den Assem}, {Martijn J.}",
year = "2018",
month = "1",
doi = "10.1038/s41562-017-0247-6",
language = "English",
volume = "2",
pages = "21--26",
journal = "Nature Human Behaviour",
issn = "2397-3374",
publisher = "Springer Nature",
number = "1",

}

The Wisdom of the Inner Crowd in Three Large Natural Experiments. / van Dolder, Dennie; van den Assem, Martijn J.

In: Nature Human Behaviour, Vol. 2, No. 1, 01.2018, p. 21-26.

Research output: Contribution to JournalArticleAcademicpeer-review

TY - JOUR

T1 - The Wisdom of the Inner Crowd in Three Large Natural Experiments

AU - van Dolder,Dennie

AU - van den Assem,Martijn J.

PY - 2018/1

Y1 - 2018/1

N2 - The quality of decisions depends on the accuracy of estimates of relevant quantities. According to the wisdom of crowds principle, accurate estimates can be obtained by combining the judgements of different individuals 1,2. This principle has been successfully applied to improve, for example, economic forecasts 3-5, medical judgements 6-9 and meteorological predictions 10-13. Unfortunately, there are many situations in which it is infeasible to collect judgements of others. Recent research proposes that a similar principle applies to repeated judgements from the same person 14. This paper tests this promising approach on a large scale in a real-world context. Using proprietary data comprising 1.2 million observations from three incentivized guessing competitions, we find that within-person aggregation indeed improves accuracy and that the method works better when there is a time delay between subsequent judgements. However, the benefit pales against that of between-person aggregation: the average of a large number of judgements from the same person is barely better than the average of two judgements from different people.

AB - The quality of decisions depends on the accuracy of estimates of relevant quantities. According to the wisdom of crowds principle, accurate estimates can be obtained by combining the judgements of different individuals 1,2. This principle has been successfully applied to improve, for example, economic forecasts 3-5, medical judgements 6-9 and meteorological predictions 10-13. Unfortunately, there are many situations in which it is infeasible to collect judgements of others. Recent research proposes that a similar principle applies to repeated judgements from the same person 14. This paper tests this promising approach on a large scale in a real-world context. Using proprietary data comprising 1.2 million observations from three incentivized guessing competitions, we find that within-person aggregation indeed improves accuracy and that the method works better when there is a time delay between subsequent judgements. However, the benefit pales against that of between-person aggregation: the average of a large number of judgements from the same person is barely better than the average of two judgements from different people.

UR - http://www.scopus.com/inward/record.url?scp=85042751115&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85042751115&partnerID=8YFLogxK

U2 - 10.1038/s41562-017-0247-6

DO - 10.1038/s41562-017-0247-6

M3 - Article

VL - 2

SP - 21

EP - 26

JO - Nature Human Behaviour

T2 - Nature Human Behaviour

JF - Nature Human Behaviour

SN - 2397-3374

IS - 1

ER -