Avoiding Overfitting: Searching for Parameter Plateau

Follow us on LinkedIn

A problem when designing a trading system is the overfitting phenomenon, wherein the system is excessively tuned to historical data. Overfitting occurs when a trading strategy performs exceptionally well on past data but fails to generalize to new, unseen data. This can lead to false positives and inflated expectations, as the system may appear profitable due to chance rather than true predictive power.

To mitigate overfitting, traders often employ techniques such as cross-validation and out-of-sample testing to ensure that their strategies remain robust across various market conditions and time periods.

Another technique to prevent overfitting involves selecting a parameter region, often referred to as a “plateau,” where the trading system maintains stable performance. Reference [1] introduced a method for quantifying this plateau and utilized particle-swarm optimization to search for it. The authors pointed out,

In this study, the concept of a parameter plateau was introduced, developing a plateau score algorithm, with the aim of replacing the conventional method of directly transferring the best-performing parameters from the training set to the testing set. The plateau score algorithm was effective in avoiding parameter islands, showcasing stable performance with a high probability. Experimental results illustrated that parameters with elevated plateau scores exhibit similar or improved performance on the testing set compared to the training set. This experimental validation underscores the substantial impact of the proposed parameter plateau definition and algorithm on parameter selection.

Subsequently, a more intricate trading strategy was examined, entailing a substantial increase in the number of parameters to be explored. In this context, unified design coupled with particle swarm optimization was employed to compute the plateau scores. An experiment encompassing the search for parameters in two- to six-dimensional trading strategies was conducted. The integration of PSO in plateau score computation significantly enhanced search efficiency compared to the brute-force method, yielding commendable final search parameters. Subsequently, an experiment involving the fine-tuning of hyperparameters for PSO in the parameter plateau was conducted. Disregarding search time considerations, a hyperparameter range was proposed for the parameter plateau applicable to PSO.

In short, the extent of plateau stability is quantified, and an efficient optimization algorithm is utilized to search for it. The out-of-sample test results show promise.

Let us know what you think in the comments below or in the discussion forum.

References

[1]  Jimmy Ming-Tai Wu, Wen-Yu Lin, Ko-Wei Huang, Mu-En Wu, On the design of searching algorithm for parameter plateau in quantitative trading strategies using particle swarm optimization, Knowledge-Based Systems, Volume 293, 7 June 2024, 111630

Further questions

What's your question? Ask it in the discussion forum

Have an answer to the questions below? Post it here or in the forum

LATEST NEWSMy grandmothers had a 24-year age gap. The difference in our relationships taught me youth and health aren't the same.
My grandmothers had a 24-year age gap. The difference in our relationships taught me youth and health aren't the same.

My grandmothers had a 24-year age gap. Our relationships were different; I was closer with the older grandma, while the younger one seemed distant.

Stay up-to-date with the latest news - click here
LATEST NEWSFlorida's Ron DeSantis is passing bills all over the place after his failed presidential campaign. They have a common theme.
Florida's Ron DeSantis is passing bills all over the place after his failed presidential campaign. They have a common theme.

Since his campaign ended, DeSantis has barred homeless people from sleeping in public and prohibited kids from social media, among other things.

Stay up-to-date with the latest news - click here
LATEST NEWSProsecutors search Peruvian president's home in graft inquiry
Prosecutors search Peruvian president's home in graft inquiry
Stay up-to-date with the latest news - click here
LATEST NEWSPeople taken hostage in Dutch night club, homes evacuated, police say
People taken hostage in Dutch night club, homes evacuated, police say
Stay up-to-date with the latest news - click here
LATEST NEWS'Spring mania' impacts people with bipolar disorder and can be life-threatening. A therapist shared how she preps for the season every year.
'Spring mania' impacts people with bipolar disorder and can be life-threatening. A therapist shared how she preps for the season every year.

People with bipolar disorder are more sensitive to physical changes in the spring. This can trigger manic episodes.

Stay up-to-date with the latest news - click here

Leave a Reply