METU Department of Economics Seminar Series (April 11th)
MIDDLE EAST TECHNICAL UNIVERSITY
Department of Economics Seminar Series
"Extracting the collective wisdom in probabilistic judgments"
(Tinbergen Institute, Erasmus University Rotterdam)
How should we combine disagreeing expert judgments on the likelihood of an event? A common solution is simple averaging, which allows independent individual errors to cancel out. However, judgments can be correlated due to an overlap in their information, resulting in a miscalibration in the simple average. Optimal weights for weighted averaging are typically unknown and require past data to estimate reliably. This paper proposes an algorithm to aggregate probabilistic judgments under shared information. Experts are asked to report a prediction and a meta-prediction. The latter is an estimate of the average of other individuals' predictions. In a Bayesian setup, I show that if average prediction is a consistent estimator, the percentage of predictions and meta-predictions that overshoot the average prediction should be the same. An "overshoot surprise" occurs when the two measures differ. The Surprising Overshoot (SO) algorithm uses the information revealed in an overshoot surprise to correct for miscalibration in the average prediction. Experimental evidence suggests that the algorithm performs well in moderate to large samples and in difficult aggregation problems where individuals often disagree in their predictions.
Date: April 11, 2022 (Monday)
Synchronous Online Seminar
Meeting ID: 962 6861 4522