I am developing a time-series forecasting model for stock market data, which is notoriously non-stationary and exhibits complex seasonality and trend components. What are the current, most effective techniques in Data Science for transforming or modeling this non-stationarity to achieve reliable predictions? I've used ARIMA in the past but am looking at more advanced Deep Learning models like LSTMs and Transformers. Does the differencing approach from traditional methods still apply, or do these modern Machine Learning algorithms inherently handle the trend and seasonality better? I need a highly accurate model with robust performance.
3 answers
Non-stationarity is the core challenge in modeling financial time-series. While traditional methods like ARIMA/SARIMA rely on explicit transformations like differencing (to achieve stationarity) or decomposing the series, modern Deep Learning architectures, especially LSTMs and Temporal Convolutional Networks (TCNs), can implicitly learn and model these non-stationary trends and long-term dependencies.However, even with these advanced models, you often still benefit from normalizing or standardizing the data. Moreover, exogenous variables and attention mechanisms (as used in Transformers) are crucial for capturing complex external factors influencing the non-stationary nature of highly volatile series like stock prices.
That makes sense, especially regarding the implicit learning of Deep Learning models. But if we are using LSTMs for forecasting, shouldn't we be extremely mindful of look-back windows? Does the size of the look-back window directly impact the model's ability to successfully capture and model the non-stationary long-term trend without introducing significant memory or computational overhead?
For non-stationary data, you must de-trend or de-season the series first, or employ Deep Learning models like LSTMs that can internally capture those long-range dependencies, ensuring better time-series forecasting accuracy.
Just to build on that, using Wavelet Transforms for de-noising before feeding the data into a Deep Learning model like an LSTM can also significantly improve the model's ability to focus on the underlying non-stationary signal and boost forecasting performance.
Samuel, the look-back window is a critical hyperparameter for LSTMs in time-series forecasting. A larger window allows the model to capture longer-term dependencies and better model the non-stationary trend; however, setting it too large increases computational cost and risks overfitting to historical noise. A pragmatic approach in Data Science is to use techniques like Partial Autocorrelation Function (PACF) to inform a good starting window size, and then systematically tune it to optimize the model's performance on the validation set, balancing accuracy with resource efficiency.