Data science and machine learning have made great progress in the past few years. They are being applied successfully in many areas such as computer vision, natural language processing, and predictive analytics.
In the financial market, however, there are still many uncertainties and risks that the new technology cannot predict. The difficulty in forecasting the financial market is due to the unpredictable nature of financial data, a low signal-to-noise ratio in available variables, and model uncertainty. Specifically, financial time series are notoriously non-stationary, and the model parameters are often unstable.
Reference [1] provided an overview of the current state of research on forecasting earnings and returns. It pointed out,
Following prior research, we highlight three major challenges for a forecaster when working with financial data: unpredictability of earnings and returns, noisy X variables, and model uncertainty. Using these challenges as a way to organize the literature, we discuss recent research that advances our collective ability to understand and predict the cross-sections of earnings and returns.
Here we reiterate some important insights from the literature. First, even with recent advancements, finding new meaningful predictors remains an important effort. Second, new out-of-the-box methods may have limited usefulness, but the thoughtful use of estimation methods and constraints seems to present promising opportunities. Third, it continues to be the case that finding earnings predictors that provide better forecasts than lagged earnings is challenging. Fourth, sorting through, combining, and understanding different models and methods likely has a long way to go before we achieve anything close to recommended best practices.
In short, forecasting the financial market is still a challenging task. In our opinion, most of the new forecasting methodologies that use machine learning were developed without good domain knowledge. It’s not a surprise that they do not perform well.
Let us know what you think in the comments below.
References
[1] Green, Jeremiah and Zhao, Wanjia, Forecasting Earnings and Returns: A Review of Recent Advancements. https://ssrn.com/abstract=4033164
Further questions
What's your question? Ask it in the discussion forum
Have an answer to the questions below? Post it here or in the forum
Gold rose to an all-time-high, after concern that President Donald Trump could fire Federal Reserve Chair Jerome Powell triggered a flight from US stocks, bonds and the dollar.
