Validity of Research in Asset Management

Follow us on LinkedIn

Academic and practitioners’ literature is a source for many investment and trading ideas. However, it is often criticized for producing ideas that do not work in practice. A problem we have seen repeatedly is that the published papers, more often than not, only present in-sample results and disregard out-of-sample ones. This is an important distinction that is often overlooked.

In-sample results are generated by using the same data to estimate a model and then using the estimated model to make predictions. This approach can produce seemingly good results, but the model may not perform well when applied to new data. Out-of-sample results are generated by using a part of the data to estimate the model and then using the estimated model to make predictions for the remaining data. This approach provides a more accurate assessment of the model’s predictive power.

Reference [1] presented an overview of the literature and it confirmed the problem of in-sample overfitting in asset management research. It also provided insights into the quality of research conducted by practitioners,

I believe p-hacking is less of a problem in asset management than in academia—in particular, less of a problem in the proprietary research that is the foundation for a product. The reasons are simple. First, in the presence of a performance fee, the asset management company’s research needs to be optimized in a way that maximizes the chances of repeatable performance (see Harvey and Liu (2018)). This means the asset manager does not choose the best-performing backtest, because likely it resulted from overfitting and luck. If the manager were to launch a backtest-overfitted strategy, it would likely fail and generate no performance fees. The second reason is reputation. Academic tenure has no equivalent in asset management. If an asset manager’s products disappoint because of overfitting, the firm’s investors will flee. This market mechanism naturally minimizes the overfitting. That said, asset management companies still produce a substantial amount of low-quality research. Similar to the academic research, investors need to be skeptical.

In short, in-sample overfitting is less of a problem in proprietary research than in academic research. Regardless, investors should be skeptical of both types of research results.

Let us know what you think in the comments below.

References

[1] Harvey, Campbell R., The Pitfalls of Asset Management Research (2022). https://ssrn.com/abstract=4078138

Further questions

What's your question? Ask it in the discussion forum

Have an answer to the questions below? Post it here or in the forum

LATEST NEWSThe South Park stock market jinx could be coming for high-flying weight-loss stocks
The South Park stock market jinx could be coming for high-flying weight-loss stocks

An analysis found that a stock underperformed the S&P 500 by a median of 7% in the year after being featured in the animated comedy.

Stay up-to-date with the latest news - click here
LATEST NEWSEarnings call: CollPlant reports Q1 2024 results and corporate updates
Earnings call: CollPlant reports Q1 2024 results and corporate updates
Stay up-to-date with the latest news - click here
LATEST NEWSEarnings call: Covalon turns profitable with strong Q2 performance
Earnings call: Covalon turns profitable with strong Q2 performance
Stay up-to-date with the latest news - click here
LATEST NEWSUS health insurer shares fall after UnitedHealth flags Medicaid enrollment issues
US health insurer shares fall after UnitedHealth flags Medicaid enrollment issues
Stay up-to-date with the latest news - click here
LATEST NEWSBESREMi ® (Ropeginterferon alfa-2B) Shows Greatest Benefit Among Cytoreductive Therapies in Lowering Symptomatic Burden of Polycythemia Vera (PV): Real-World Analysis Published at ASCO
BESREMi ® (Ropeginterferon alfa-2B) Shows Greatest Benefit Among Cytoreductive Therapies in Lowering Symptomatic Burden of Polycythemia Vera (PV): Real-World Analysis Published at ASCO
Stay up-to-date with the latest news - click here

Leave a Reply