The weathermen were asked to predict the weather based on weather patterns they were given and the doctors had to diagnose a patient based on conditions they were given. The doctors were given the task to make a diagnosis based on relevant information. Both groups were subsequently asked to state their level of confidence.

The interesting outcome was that there was a near perfect correlation between the level of confidence given by the weathermen and the accuracy of their forecasts. Contrary to this, the doctors severely overestimated the quality of their diagnosis. When they were 90% sure to be right, they were actually correct in only 15% of the cases.

As stunning as these findings are, they don't come as a surprise. Weathermen benefit from very rapid feedback loops. Whether a forecast turned out correct or not can quickly and objectively be assessed. The evaluation of medical diagnosis is a different and way more complex task.

Unfortunately as another survey has found, financial market professionals tend to be closer to doctors than to weathermen.

This study asked, both psychology students and investment professional (portfolio managers, analysts and brokers) to provide forecast which stocks would outperform and to state their level of confidence. Interestingly, both groups overestimated their ability to forecast stock prices, indicating that they suffered from overconfidence.

However, overconfidence was much more pronounced among the professionals. In other words, investment experience did not lead to an increase in judgement ability but rather to an increase in arrogance.

In our view, the problem is that feedback loops in investing are not easily established. Weathermen are required to provide objective forecasts for a given period of time followed by near instant feedback. However, in stock picking it takes a large number of bets and substantial period of time to tell skill from luck and it is tempting for professionals to ignore the feedback given by the market completely.

That's why it is even more important to establish processes that enable analyst and portfolio managers to learn from their mistakes.

Our paper elaborates on the topic in the context of single stock valuation with automated and upgradeable Discounted Cashflow Models.