Avoiding Overfitting: Searching for Parameter Plateau

Follow us on LinkedIn

A serious problem when designing a trading system is the overfitting phenomenon, wherein the system is excessively tuned to historical data. Overfitting occurs when a trading strategy performs exceptionally well on past data but fails to generalize to new, unseen data. This can lead to false positives and inflated expectations, as the system may appear profitable due to chance rather than true predictive power.

To mitigate overfitting, system developers often employ techniques such as cross-validation and out-of-sample testing to ensure that their strategies remain robust across various market conditions and time periods.

Another technique to prevent overfitting involves selecting a parameter region, often referred to as a “plateau,” where the trading system maintains stable performance. Reference [1] introduced a method for quantifying this plateau and utilized particle-swarm optimization to search for it. The authors pointed out,

In this study, the concept of a parameter plateau was introduced, developing a plateau score algorithm, with the aim of replacing the conventional method of directly transferring the best-performing parameters from the training set to the testing set. The plateau score algorithm was effective in avoiding parameter islands, showcasing stable performance with a high probability. Experimental results illustrated that parameters with elevated plateau scores exhibit similar or improved performance on the testing set compared to the training set. This experimental validation underscores the substantial impact of the proposed parameter plateau definition and algorithm on parameter selection.

Subsequently, a more intricate trading strategy was examined, entailing a substantial increase in the number of parameters to be explored. In this context, unified design coupled with particle swarm optimization was employed to compute the plateau scores. An experiment encompassing the search for parameters in two- to six-dimensional trading strategies was conducted. The integration of PSO in plateau score computation significantly enhanced search efficiency compared to the brute-force method, yielding commendable final search parameters. Subsequently, an experiment involving the fine-tuning of hyperparameters for PSO in the parameter plateau was conducted. Disregarding search time considerations, a hyperparameter range was proposed for the parameter plateau applicable to PSO.

In short, the extent of plateau stability is quantified, and an efficient optimization algorithm is utilized to search for it. The out-of-sample test results show promise.

Let us know what you think in the comments below or in the discussion forum.

References

[1]  Jimmy Ming-Tai Wu, Wen-Yu Lin, Ko-Wei Huang, Mu-En Wu, On the design of searching algorithm for parameter plateau in quantitative trading strategies using particle swarm optimization, Knowledge-Based Systems, Volume 293, 7 June 2024, 111630

Further questions

What's your question? Ask it in the discussion forum

Have an answer to the questions below? Post it here or in the forum

LATEST NEWSMassive international police operation takes down ransomware networks, arrests 4 suspects
Massive international police operation takes down ransomware networks, arrests 4 suspects

THE HAGUE, Netherlands (AP) — Police coordinated by the European Union’s justice agency have taken down computer networks responsible for spreading ransomware via infected emails, in what they called the biggest ever international operation against the lucrative form of cybercrime. The European Union’s judicial cooperation…

Stay up-to-date with the latest news - click here
LATEST NEWSFourteen Hong Kong democrats found guilty in landmark subversion trial
Fourteen Hong Kong democrats found guilty in landmark subversion trial
Stay up-to-date with the latest news - click here
LATEST NEWSIndia's home prices to rise steadily, affordable housing supply to lag demand: Reuters poll
India's home prices to rise steadily, affordable housing supply to lag demand: Reuters poll
Stay up-to-date with the latest news - click here
LATEST NEWSAs Europe struggles to cut debt, Sweden eyes more spending
As Europe struggles to cut debt, Sweden eyes more spending
Stay up-to-date with the latest news - click here
LATEST NEWSNxera Pharma to Receive $4.6 Million in Milestone Payments from Centessa Pharmaceuticals
Nxera Pharma to Receive $4.6 Million in Milestone Payments from Centessa Pharmaceuticals
Stay up-to-date with the latest news - click here

Leave a Reply