Formal Study of Overfitting in Trading System Design

Subscribe to newsletter

Trading systems often experience performance deterioration after going live, largely due to overfitting. Reference [1] formally studied this issue, using analytical approximations for the in-sample and out-of-sample Sharpe ratios of portfolios. The authors pointed out,

This paper derives analytical approximations for the in-sample and out-of-sample Sharpe ratios of portfolios constructed using linear prediction models. We show that increasing either the number of signals or assets too much makes this procedure susceptible to overfitting and thereby yields wildly overestimated in-sample Sharpe ratios.

We show that low true Sharpe ratio signals are particularly vulnerable to overfitting. Conversely, by extending the length of the in-sample period one can reduce the overfitting risk, and can produce a higher replication ratio out of sample.

Subscribe to newsletter https://harbourfrontquant.beehiiv.com/subscribe Newsletter Covering Trading Strategies, Risk Management, Financial Derivatives, Career Perspectives, and More

We test our results on commodity futures using momentum-style signals and find that allowing AR(1) signals and non-Normal signals/residuals does not significantly impact the validity of our results. In particular, once we match the theoretical out-of-sample Sharpe ratio to the observed value, we see that the replication ratio is primarily a function of the out-of-sample Sharpe ratio and the curves for the AR(1) signals closely matches those of iid signals.

From this analysis, it seems that the best way to minimize the potential for overfitting is to minimize the number of signals and assets that are being used for any predictive model used to trade, and utilize the largest amount of data possible.

In summary, the paper formally demonstrated that to minimize the risk of overfitting, one should,

  1. Keep models as simple as possible,
  2. Use the longest sensible backtest period available,
  3. Develop systems with high Sharpe ratios, and
  4. Rely on fewer signals.

While we completely agree with points #1 and #2, our experience casts doubt on points #3 and #4.

Let us know what you think in the comments below or in the discussion forum.

References

[1] Antoine Jacquier, Johannes Muhle-Karbe, Joseph Mulligan, In-Sample and Out-of-Sample Sharpe Ratios for Linear Predictive Models, 2025, arXiv:2501.03938

Further questions

What's your question? Ask it in the discussion forum

Have an answer to the questions below? Post it here or in the forum

LATEST NEWSTech-Focused Tiger Cubs Notch January Gains Amid DeepSeek Drama
Tech-Focused Tiger Cubs Notch January Gains Amid DeepSeek Drama

Hedge funds run by proteges of the late Julian Robertson beat the broader equity markets in January, shrugging off gyrations sparked by the emergence of Chinese AI upstart DeepSeek and threats of tariffs.

Stay up-to-date with the latest news - click here
LATEST NEWSIntuitive Surgical's CFO sells shares worth $158,710
Intuitive Surgical's CFO sells shares worth $158,710
Stay up-to-date with the latest news - click here
LATEST NEWSTrump says end Gaza ceasefire by Sat if hostages not released
Trump says end Gaza ceasefire by Sat if hostages not released
Stay up-to-date with the latest news - click here
LATEST NEWSSeeking AI boom, France and EU promise to cut red tape on tech
Seeking AI boom, France and EU promise to cut red tape on tech
Stay up-to-date with the latest news - click here
LATEST NEWSKimbal Musk sells Tesla stock worth $27.6 million
Kimbal Musk sells Tesla stock worth $27.6 million
Stay up-to-date with the latest news - click here

Leave a Reply