There are two fundamental truths of prediction: (i) there exists a “prediction horizon” where predictions beyond this point are inherently inaccurate; and (ii) experts are generally just bad at predicting.
On that first point:
our ability to predict is limited by the nature of complex systems. Weather forecasts, for example, are quite accurate a day or two out. Three or four days out, they are less accurate. Beyond a week, we might as well flip a coin. As scientists learn more about weather, and computing power and sophistication grow, this forecasting horizon may be pushed out somewhat. But there will always be a point beyond which meteorologists cannot see, even in theory.
Prediction horizons vary, but the general idea is the same whether experts are trying to forecast the weather, economies, elections or social unrest: No matter how brilliant the analysts may be, no matter how abundant the resources at their disposal, their vision can only go so far.
Yet, despite these inherent limitations, there is hope.
In Expert Political Judgment, Philip Tetlock’s seminal work on political forecasting, he shows that many experts would do better if they simply guessed randomly. However, his research also offers guidance on how to improve our forecasting accuracy:
what separated those with modest but significant predictive ability from the utterly hopeless was their style of thinking. Experts […] were handily beaten by those who used diverse information and analytical models, were comfortable with complexity and uncertainty and kept their confidence in check.
What this and much other research suggests is that the right training, tools and organization can make people better forecasters.
Based on these findings, Tetlock received IARPA funding to explore this further (as the ACE Program). His research culminated in The Good Judgement Project (a crowd forecasting organisation) and the excellent Superforecasting. I’ll dive more into this topic in the days to come.