Data science and machine learning are advancing at a rapid pace. They’re now being applied in areas as diverse as healthcare, retail, marketing, and finance. However, a key question that still needs to be answered is: how much data do you need to train these models?
The answer, it turns out, is not always more data. In some cases, using too much data can actually hurt the performance of your machine learning models. In this context, Reference  argued that more data is not always better,
Managers often believe that collecting more data will continually improve the accuracy of their machine learning models. However, we argue in this paper that when data lose relevance over time, it may be optimal to collect a limited amount of recent data instead of keeping around an infinite supply of older (less relevant) data. In addition, we argue that increasing the stock of data by including older datasets may, in fact, damage the model’s accuracy. Expectedly, the model’s accuracy improves by increasing the flow of data (defined as data collection rate); however, it requires other tradeoffs in terms of refreshing or retraining machine learning models more frequently.
The paper also pointed out that the value of a firm does not scale with its stock of data,
This result, coupled with the fact that older datasets may deteriorate models’ accuracy, suggests that created business value doesn’t scale with the stock of available data unless the firm offloads less relevant data from its data repository. Consequently, a firm’s growth policy should incorporate a balance between the stock of historical data and the flow of new data.
What implication does this paper have for trading and portfolio management? Should we use more data?
The short answer is probably no. In fact, using more data can actually lead to sub-optimal results. The reason is that, in the financial world, data is often noisy and contains a lot of irrelevant information. If you use too much data, your machine learning models will end up picking up on this noise, which can lead to sub-optimal results.
So how do we use data for trading? Let us know in the comments below.
 Valavi, Ehsan, Joel Hestness, Newsha Ardalani, and Marco Iansiti. Time and the Value of Data. Harvard Business School Working Paper, No. 21-016, August 2020. (Revised November 2021.)
Have an answer to the questions below? Post it here or in the forum
BRUSSELS — EU countries will seek a deal on Friday on a European Commission proposal to set price caps on Russian oil products, after postponing a decision on Wednesday amid divisions between member states, diplomats said. The European Commission proposed last week that from Feb.…
WASHINGTON, Feb 1 (Reuters) – President Joe Biden on Wednesday unveiled fresh efforts to slash credit card late fees and drive down the prices that Apple Inc and Google parent Alphabet Inc charge on mobile app stores. The move is part of a larger policy…
NEW YORK — Subway’s late co-founder has left his 50% stake in the sandwich chain to his philanthropic foundation, protecting billions of dollars in proceeds from the company’s potential sale from the reach of the U.S. tax authorities. The Peter and Carmen Lucia Buck Foundation…
NEW YORK (AP) — Former President Donald Trump’s political operation started the year with about $25 million socked away for his recently launched 2024 presidential campaign, a sum that is substantially less than what he had on hand just one year ago, new campaign finance…