Friday, June 13, 2014

Elite Forecasters and The Best Way to Predict the Future

A topic of abiding interest.
From the BBC:

(Getty Images)
A small group of people have a surprising knack for correctly predicting the course of 
world events – and you could be one of them, says David Robson. (Getty Images)
Cast your mind back across the turbulent events of recent history. Did you foresee President Obama’s election before he was even elected as a Democratic candidate – or did you back Hillary Clinton? How about the Arab Spring – could you hear the revolution in the first tremors of dissatisfaction? And did you faithfully predict the recent Ukraine crisis?

If you answer yes to these questions, you could be a “super-forecaster”, someone who is able to foresee the outcome of world events with astonishing accuracy. This has nothing to do with the reading of tea leaves; nor do you have to be a seasoned political pundit. Some super-forecasters are just everyday people, from all walks of life, who happen to have astonishingly perceptive judgements.

Over the last few years, psychologists have found a series of “hidden” talents for extraordinary mental skills that can go unnoticed by the people harbouring them. There are “super-recognisers”, for instance, who can easily remember people’s faces, even if they had only met the person once, years before. Other gifted individuals include super-tasters, who have an intense perception of flavour and super memorisers who naturally recall events from almost every day of their lives.

Those talents might be largely down to our genes, whereas forecasting ability should, in theory, be down to experience and learned skill. Yet top political experts tend to perform very poorly at crystal ball gazing: Philip Tetlock at the University of Pennsylvania has found they do only slightly better than chance. “Chimps randomly throwing darts at the possible outcomes would have done almost as well as the experts,” is how one political scientist summarised the findings to the New York Times.
Intelligence agencies have a keen interest in research on forecasting (Getty Images)
Needless to say, military intelligence agencies have been watching closely. Inspired, in part, by one of Tetlock’s reports, the US Intelligence Advanced Research Projects Activity (IARPA) sponsored the four-year-long “Good Judgement Project” that aimed to find new approaches to political forecasting. Taking the form of a tournament, it recruited thousands of participants from all walks of life to test their forecasting skills on a range of questions, such as “Will Robert Mugabe cease to be President of Zimbabwe by 30 September 2011?” or “Will Greece remain a member of the EU through 1 June 2012?” Rather than simple yes or no answers – which cannot reflect the inherent uncertainty of life – they were asked to estimate the probability that each event would occur.

Three years into the tournament, Tetlock’s team has now released some of its first results to the journal Psychological Science, and at the annual meeting of the Association for Psychological Science in San Francisco last month.

One of the main aims was to see whether certain forecasters were consistently good. So after one year Tetlock checked the results of more than 2,000 participants and siphoned off the top 2% – the so called “super forecasters”, who then formed teams for the rest of the tournament. At the end of the second year, their performance was up to four times more accurate than other participants.

Hidden talents
Although political expertise may have helped, they came from a variety of professions. “One of the best forecasters was a pharmacist,” says Tetlock.

As you might expect, these elite forecasters tended to score better on measures of intelligence than the other participants. But they all shared one other trait too: open-mindedness. In everyday life, open-mindedness may be mistaken for having liberal political views, but in psychology it is thought to reflect how well you deal with uncertainty. Crucially, open-minded people tend to be able to see problems from all sides, which seems to help forecasters overcome their preconceptions in the light of new evidence. “You need to change your mind fast, and often,” says Tetlock....MORE
We have many, many posts on the subject, if interested use the search blog box with the keyword forecasting.
Or, if that's not exactly the direction you want to go you may wish to consider:
"Pseudo-Mathematics and Financial Charlatanism...."

And previously on the Mountebank channel:
UPDATED--Are You a Recent Graduate Who Hasn't Found a Job? Consider Becoming a Charlatan
Follow-up: Choosing the Charlatan Career Path
Re-post: Peak Oil Stalwart to Shutter Forum/News Site, Persue Career as Astrologer
See also:
Technical analysis
Fundamental analysis
Divination for Dummies
Pitfalls in Prognostication: Fortune Magazine's August, 2000 "Ten Stocks to Last the Decade"