TY - JOUR
T1 - Insights into the accuracy of social scientists’ forecasts of societal change
AU - Grossmann, Igor
AU - Rotella, Amanda
AU - Hutcherson, Cendri A.
AU - Sharpinskyi, Konstantyn
AU - Varnum, Michael E. W.
AU - Achter, Sebastian
AU - Dhami, Mandeep K.
AU - Guo, Xinqi Evie
AU - Kara-Yakoubian, Mane
AU - Mandel, David R.
AU - Raes, Louis
AU - Tay, Louis
AU - Vie, Aymeric
AU - Wagner, Lisa
AU - Adamkovic, Matus
AU - Arami, Arash
AU - Arriaga, Patrícia
AU - Bandara, Kasun
AU - Baník, Gabriel
AU - Bartoš, František
AU - Baskin, Ernest
AU - Bergmeir, Christoph
AU - Białek, Michał
AU - Børsting, Caroline K.
AU - Browne, Dillon T.
AU - Caruso, Eugene M.
AU - Chen, Rong
AU - Chie, Bin-Tzong
AU - Chopik, William J.
AU - Collins, Robert N.
AU - Cong, Chin Wen
AU - Conway, Lucian G.
AU - Davis, Matthew
AU - Day, Martin V.
AU - Dhaliwal, Nathan A.
AU - Durham, Justin D.
AU - Dziekan, Martyna
AU - Elbaek, Christian T.
AU - Shuman, Eric
AU - Fabrykant, Marharyta
AU - Firat, Mustafa
AU - Fong, Geoffrey T.
AU - Frimer, Jeremy A.
AU - Gallegos, Jonathan M.
AU - Goldberg, Simon B.
AU - Gollwitzer, Anton
AU - Goyal, Julia
AU - Graf-Vlachy, Lorenz
AU - Gronlund, Scott D.
AU - Karg, Simon T.
AU - The Forecasting Collaborative
PY - 2023
Y1 - 2023
N2 - How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender–career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists’ forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data.
AB - How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender–career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists’ forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data.
KW - Forecasting
KW - Humans
KW - Models, Statistical
U2 - 10.1038/s41562-022-01517-1
DO - 10.1038/s41562-022-01517-1
M3 - Journal article
C2 - 36759585
SN - 2397-3374
VL - 7
SP - 484
EP - 501
JO - Nature Human Behaviour
JF - Nature Human Behaviour
IS - 4
ER -