The Costly Mistake of Misjudging Luck from Skill

Good investment performance tends to attract assets. Investors are inclined to chase returns as they attribute a fund manager’s outperformance, even over short time horizons, to skill. However, by neglecting the role of randomness in track records, many investors assume that past performance is a good predictor of future performance. That’s a problem and this article explains why.

Separating skill from luck: The importance of sample size and time

Trading results include both skill and luck. When analyzing small sample sizes, skill is hard to detect. An outperforming trader could be an unskilled trader on a lucky streak. Likewise, an underperforming trader may be suffering through a period of bad luck or indeed be lacking an edge. Separating skill from luck isn’t easy.

Case A: Pure luck (e.g., rolling a fair die, tossing a fair coin)

For a large enough sample, the tossing of a fair coin produces a 50/50 win-loss ratio for heads vs. tails.

Case B: More luck than skill

An unskilled trader can increase this 50/50 coin-toss-ratio (say, to 90/10) by selling far out-of-the-money equity put options, for example. Most investors prefer high winning percentages over low winning percentages because it feels good to win most of the time — which is why these strategies are attractive to clients. However, in most cases this means investors are turning a blind eye to the large downside risks hidden in the tail of a negatively skewed return distribution. While drawdowns for these strategies may be rare, the losses can be disastrous when they finally occur. The last week is a case in point. Trading systems which lack robustness tend to disintegrate when the market environment changes.

In this case, when chasing recent returns, a too short sample period may cause investors to incorrectly conclude that skill (alpha) is the dominating return source. Even track records spanning decades may be too short.

Case C: More skill than luck

Many skilled traders have winning percentages which are worse than 50/50. For example, the long-term winning percentages of most trend following trading systems is about 40%. Compared to selling out-of-the-money equity puts, this 40/60 ratio doesn’t feel so good: Investors lose more often than they win and drawdowns are the norm rather than the exception. However, trend following trading systems have a real edge, as demonstrated by the many CTA track records which go back 30-40 years and sometimes even more. The last week was a reminder that trend following trading systems can be very volatile and that drawdowns can happen fast. But these systems are robust and can stand the test of time, which is why their drawdowns are seldom disastrous.

In this case, when chasing recent returns, a too short sample period may cause investors to overlook the presence of skill in the trader’s track record.

Returning to case A, when tossing a coin many times (e.g., 1,000 tosses), the regression to the mean will cause the actual win-loss ratio to be very close to 50/50. But tossing the coin a mere 10 times may produce a very different outcome due to the randomness in small sample sizes. Hence, luck dominates the outcome in the short-run, but the actual probabilities will dominate it in the long-run.

Regarding case B, a manager may claim to outperform the DAX by 4% each year while staying 90% correlated to the index and realizing approximately the same volatility. In an attempt to achieve this result, the manager decides to sell far out-of-the-money DAX put options on a monthly basis. We can conduct a t-test to approximate how much data the investor will have to examine to see through the randomness in the manager’s return stream: To be 95% confident that the manager’s outperformance is indeed the result of a true trading edge, the investor would have to analyze more than for 13.4 years of performance data. Not all managers have that long a track record, and most investors don’t have that much time. The issue here is a low signal-to-noise ratio in the manager’s trading results. It may cause the investor to detect skill where in fact none is present.

With respect to case C, consider the example of a trend following “Golden Cross” trading system (50/200 day moving average crossover). During the past 20 years, this system has produced about 41% winning trades and 59% losing trades for a portfolio of 60 different futures markets. However, the average gain per trade in dollar terms is 2.55 times larger than the average loss, and thus the system’s expectancy per trade is (41%*$2.55) — (59%*$1) = $0.45. From a probability standpoint it’s not at all unlikely for this trading system to have six losing trades in a row (or even more).

Even though this system tends to earn $0.45 per trade on average in the long-run, pure bad luck can cause a large drawdown in the short-run. In contrast, the option selling system in case B may have a negative PL expectation in the long run, but is unlikely to be in a drawdown in the short-run.

Conclusion

Chasing returns tends to be an expensive exercise for investors. Once the investment is made, many outperforming managers tend to perform much worse than their recent performance record would suggest. That’s especially true when selecting the best performing managers from a very large group of managers. The larger the group, the greater the chance that the best performing managers were simply lucky and vice versa. Put differently, the more pair of dice you roll, the greater the probability of rolling snake eyes.

All trading results are a combination of skill and luck, but decomposing the outcome into these two categories isn’t easy. It’s impossible if the sample size is too small. By placing too much emphasis on recent returns, investors are at risk of allocating money to unskilled lucky traders while disregarding skilled but temporarily unlucky traders. This is a costly mistake.