AI Aging: Model Quality Degradation

Follow us on LinkedIn

Artificial Intelligence (AI) and Machine Learning (ML) are two rapidly growing fields that have revolutionized the way we process and analyze data. AI refers to the development of computer systems that can perform tasks that typically require human intelligence, such as visual perception, speech recognition, decision-making, and natural language processing. ML, on the other hand, is a subset of AI that involves the use of algorithms and statistical models to enable computers to learn from data without being explicitly programmed.

In the financial industry, AI and ML have become essential tools for data analysis, risk management, fraud detection, and investment decision-making. AI and ML can be used to analyze market trends, optimize portfolio management, predict stock prices, and automate trading strategies. Furthermore, AI and ML can be used to improve customer experience by enabling personalized financial advice and automated customer service.

Although AI and ML have shown tremendous potential for transforming various industries, they are not without their challenges and limitations. Reference [1] discussed the issue of AI aging, i.e. temporal quality degradation in AI models. The authors pointed out,

As AI models continue to advance into many real-life applications, their ability to maintain reliable quality over time becomes increasingly important. The principal challenge in this task stems from the very nature of current machine learning models, dependent on the data as it was at the time of training. In this study, we present the first analysis of AI “aging”: the complex, multifaceted phenomenon of AI model quality degradation as more time passes since the last model training cycle. Using datasets from four different industries (healthcare operations, transportation, finance, and weather) and four standard machine learning models, we identify and describe the main temporal degradation patterns. We also demonstrate the principal differences between temporal model degradation and related concepts that have been explored previously, such as data concept drift and continuous learning. Finally, we indicate potential causes of temporal degradation, and suggest approaches to detecting aging and reducing its impact.

Although the paper demonstrated the existence of AI model degradation in various industries, we note that the problem of model degradation is well-known in finance for decades due to the non-stationary nature of financial time series and constantly changing market conditions.

The authors also highlighted the importance of model retraining in order to cope with degradation. Specifically, we would need to

  • Develop a trigger to signal when the model must be retrained,
  • Develop an efficient and robust mechanism for automatic model retraining.

These problems are also well-known in finance and the solution is not trivial.

Let us know what you think in the comments below or in the discussion forum.

References

[1] Daniel Vela, Andrew Sharp, Richard Zhang, Trang Nguyen, An Hoang & Oleg S. Pianykh, Temporal quality degradation in AI models, Sci Rep 12, 11654 (2022). https://doi.org/10.1038/s41598-022-15245-z

Further questions

What's your question? Ask it in the discussion forum

Have an answer to the questions below? Post it here or in the forum

LATEST NEWSWall Street is in an all-out war for AI talent as banks poach each others’ employees—and Goldman Sachs is losing
Wall Street is in an all-out war for AI talent as banks poach each others’ employees—and Goldman Sachs is losing

Workers with data, analytics, or AI experience are a hot commodity, taking home a median $900,000 last year.

Stay up-to-date with the latest news - click here
LATEST NEWSBitcoin is on track to hit $100,000 by the end of 2024 as 'crypto spring' has begun, Standard Chartered says
Bitcoin is on track to hit $100,000 by the end of 2024 as 'crypto spring' has begun, Standard Chartered says

The increased likelihood of US spot ETFs for crypto means bitcoin could even reach $100,000 before the end of 2024, analysts said.

Stay up-to-date with the latest news - click here
LATEST NEWSI'm a freshman at Yale and just went to my first Harvard-Yale football game. The weekend was one of my favorites at college so far.
I'm a freshman at Yale and just went to my first Harvard-Yale football game. The weekend was one of my favorites at college so far.

At the famous Harvard-Yale football game, I tailgated and even stripped in the stadium as part of a strange tradition. I loved every moment of it.

Stay up-to-date with the latest news - click here
LATEST NEWSA tattoo artist who refuses to ink names of people's significant others shares 3 alternatives that won't end in regret
A tattoo artist who refuses to ink names of people's significant others shares 3 alternatives that won't end in regret

Mary Minahan said she's had clients who got tattoos of their partner's name return a few weeks later to have them covered up after a breakup. So now she won't do them anymore.

Stay up-to-date with the latest news - click here
LATEST NEWSIt's the spendiest season of the year—but half of Americans are buckling down to save for a safety net
It's the spendiest season of the year—but half of Americans are buckling down to save for a safety net

It's a sign they're still unhappy with the economy.

Stay up-to-date with the latest news - click here

Leave a Reply