Abstract
The quality of decisions depends on the accuracy of estimates of relevant quantities. According to the wisdom of crowds principle, accurate estimates can be obtained by combining the judgements of different individuals 1,2. This principle has been successfully applied to improve, for example, economic forecasts 3-5, medical judgements 6-9 and meteorological predictions 10-13. Unfortunately, there are many situations in which it is infeasible to collect judgements of others. Recent research proposes that a similar principle applies to repeated judgements from the same person 14. This paper tests this promising approach on a large scale in a real-world context. Using proprietary data comprising 1.2 million observations from three incentivized guessing competitions, we find that within-person aggregation indeed improves accuracy and that the method works better when there is a time delay between subsequent judgements. However, the benefit pales against that of between-person aggregation: the average of a large number of judgements from the same person is barely better than the average of two judgements from different people.
| Original language | English |
|---|---|
| Pages (from-to) | 21-26 |
| Journal | Nature Human Behaviour |
| Volume | 2 |
| Issue number | 1 |
| Early online date | 11 Dec 2017 |
| DOIs | |
| Publication status | Published - Jan 2018 |
Fingerprint
Cite this
}
The Wisdom of the Inner Crowd in Three Large Natural Experiments. / van Dolder, Dennie; van den Assem, Martijn J.
In: Nature Human Behaviour, Vol. 2, No. 1, 01.2018, p. 21-26.Research output: Contribution to Journal › Article › Academic › peer-review
TY - JOUR
T1 - The Wisdom of the Inner Crowd in Three Large Natural Experiments
AU - van Dolder, Dennie
AU - van den Assem, Martijn J.
PY - 2018/1
Y1 - 2018/1
N2 - The quality of decisions depends on the accuracy of estimates of relevant quantities. According to the wisdom of crowds principle, accurate estimates can be obtained by combining the judgements of different individuals 1,2. This principle has been successfully applied to improve, for example, economic forecasts 3-5, medical judgements 6-9 and meteorological predictions 10-13. Unfortunately, there are many situations in which it is infeasible to collect judgements of others. Recent research proposes that a similar principle applies to repeated judgements from the same person 14. This paper tests this promising approach on a large scale in a real-world context. Using proprietary data comprising 1.2 million observations from three incentivized guessing competitions, we find that within-person aggregation indeed improves accuracy and that the method works better when there is a time delay between subsequent judgements. However, the benefit pales against that of between-person aggregation: the average of a large number of judgements from the same person is barely better than the average of two judgements from different people.
AB - The quality of decisions depends on the accuracy of estimates of relevant quantities. According to the wisdom of crowds principle, accurate estimates can be obtained by combining the judgements of different individuals 1,2. This principle has been successfully applied to improve, for example, economic forecasts 3-5, medical judgements 6-9 and meteorological predictions 10-13. Unfortunately, there are many situations in which it is infeasible to collect judgements of others. Recent research proposes that a similar principle applies to repeated judgements from the same person 14. This paper tests this promising approach on a large scale in a real-world context. Using proprietary data comprising 1.2 million observations from three incentivized guessing competitions, we find that within-person aggregation indeed improves accuracy and that the method works better when there is a time delay between subsequent judgements. However, the benefit pales against that of between-person aggregation: the average of a large number of judgements from the same person is barely better than the average of two judgements from different people.
UR - http://www.scopus.com/inward/record.url?scp=85042751115&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85042751115&partnerID=8YFLogxK
U2 - 10.1038/s41562-017-0247-6
DO - 10.1038/s41562-017-0247-6
M3 - Article
VL - 2
SP - 21
EP - 26
JO - Nature Human Behaviour
JF - Nature Human Behaviour
SN - 2397-3374
IS - 1
ER -