©Author | Light
School | University of Chinese Academy of Sciences
Research Direction | Machine Learning
Zhihu Link: https://zhuanlan.zhihu.com/p/471014006
Editor| paperweekly
Time series is one of my favorite research topics. Here I list the most commonly used methods for time series, including both theory and practice. The theoretical part mainly consists of high-quality interpretations created by various experts, which I will briefly present here along with links. The practical part includes high-quality open-source code to help everyone get started quickly. Finally, I will include links to some classic solutions from Kaggle competitions related to time series for your reference and study.
Time series problems are regarded as regression problems, with differences in the regression methods (linear regression, tree models, deep learning, etc.).
Traditional Time Series ModelingThe ARIMA model is an upgraded version of the ARMA model; the ARMA model can only model stationary data, while the ARIMA model requires differencing the data before modeling. The problems that these two models can handle are relatively simple, mainly due to the following two reasons:
- The ARMA/ARIMA models are fundamentally simple linear models, limited in the complexity of problems they can represent;
- The full name of ARMA is Autoregressive Moving Average model, which can only support regression on univariate historical data and cannot handle multivariate situations.
Theory Section:A Financial Time Series Analysis: Basics
Focuses on basic financial time series knowledge and the ARMA model
https://zhuanlan.zhihu.com/p/38320827Introduction to Financial Time Series [Final Chapter] ARCH, GARCH
Introduces more advanced ARCH and GARCH models
https://zhuanlan.zhihu.com/p/21962996Practical Section:[Time Series Analysis] ARMA Prediction of GDP in Python
Quick start with the ARMA model
https://zhuanlan.zhihu.com/p/54799648machinelearningmastery.com
Quick modeling with ARCH and GARCH models
https://machinelearningmastery.com/develop-arch-and-garch-models-for-time-series-forecasting-in-python/
Summary:If dealing with univariate prediction problems, traditional time series models can be very advantageous; however, if the problems or variables are too many, traditional time series models become inadequate.
Machine Learning Model MethodsThese methods are represented by LightGBM and XGBoost, generally transforming time series problems into supervised learning, predicting through feature engineering and machine learning methods; such models can solve most complex time series prediction models. They support complex data modeling, multivariate collaborative regression, and nonlinear problems.However, this method requires a relatively complex manual feature engineering process, which needs a certain level of expertise or rich imagination. The quality of feature engineering often determines the upper limit of machine learning, and machine learning methods only strive to approach this limit. Once the features are well established, tree model algorithms like LightGBM/XGBoost can be directly applied, which are very common rapid modeling methods. In addition, they have the following characteristics:
- Fast computation speed and high model accuracy;
- No need to handle missing values, which is quite convenient;
- Support for categorical variables;
-
Support for feature cross.
Theory Section:Deep Dive into LightGBM Principles:
LightGBM principles
https://blog.csdn.net/anshuai_aw1/article/details/83659932
XGBoost Principles Are Not as Difficult as You Think:
XGBoost principles
https://www.jianshu.com/p/7467e616f227
Practical Section:Using LightGBM in Python:
LightGBM model practice
https://zhuanlan.zhihu.com/p/52583923The Most Detailed XGBoost Practical Guide:
XGBoost model practice
https://zhuanlan.zhihu.com/p/31182879Summary:After a series of feature engineering, directly using machine learning methods can solve most complex time series problems; however, the biggest drawback of this method is that feature engineering can be quite cumbersome.
Deep Learning Model MethodsThese methods primarily include LSTM/GRU, seq2seq, Wavenet, 1D-CNN, and transformer. LSTM/GRU models in deep learning are specifically designed to solve time series problems; however, CNN models, originally designed for image problems, have also evolved to solve time series issues. In general, deep learning models have the following characteristics:
- Cannot include missing values; they must be filled; otherwise, an error will occur;
- Support for feature cross, such as second-order and high-order crosses;
- Requires an embedding layer to process categorical variables, allowing direct learning of the semantic variables of discrete features and representing their relative relationships;
- When the data volume is small, the model performance is not as good as tree methods; however, when the data volume is huge, neural networks perform better;
- Neural network models support online training.
In fact, based on practical prediction problems, various deep learning model architectures can be designed. For example, if we predict a time series problem (like predicting heart rate) that is not only related to statistical data but also to text (like physician opinions) and images (like ECGs), we can combine MLP, CNN, BERT, etc., to build a more powerful model.
▲ Image Source: https://www.kaggle.com/c/avito-demand-prediction/discussion/59880
Theory Section:[Essentials] In-Depth LSTM and Its Python Implementation:
LSTM principles
https://zhuanlan.zhihu.com/p/104475016Detailed Explanation of Seq2Seq Principles – Early Morning Bug – Blog Garden:
Seq2Seq principles
https://www.cnblogs.com/liuxiaochong/p/14399416.htmlWavenet Principles and Implementation:
Wavenet principles
https://zhuanlan.zhihu.com/p/28849767How CNN Convolutional Neural Networks Handle One-Dimensional Time Series Data:
1D-CNN handling time series data
https://www.ai8py.com/cnn-in-keras-for-time-sequences.htmlTransformer for Time Series Detailed Prediction Algorithm:
Transformer time series prediction
https://zhuanlan.zhihu.com/p/391337035Practical Section:Python Implementation of the Seq2Seq Model – Natural Language Processing Application Based on Seq2Seq ModelUsage:
Seq2Seq model implementation
https://dataxujing.github.io/seq2seqlearn/chapter3/machinelearningmastery.com:
LSTM practice
https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-pythConv1d-WaveNet-Forecast Stock Price:
Wavenet model predicting stock prices
https://www.kaggle.com/bhavinmoriya/conv1d-wavenet-forecast-stock-pricetowardsdatascience.com/:
Transformer time series prediction data
https://towardsdatascience.com/how-to-use-transformer-networks-to-build-a-forecasting-model-297f9270e630
Keras documentation:
Time series classification with a Transformer model: Transformer handling time series data classification
https://keras.io/examples/timeseries/timeseries_transformer_classification/kaggle.com/fatmakursun/:
CNN prediction model
https://www.kaggle.com/fatmakursun/predict-sales-time-series-with-cnnSummary:Deep learning models can solve almost all time series problems, and the models can automatically learn feature engineering, greatly reducing manual effort; however, they require a high level of model architecture capability.Finally, I will also include some classic data mining competition links and solutions. If you can understand the data and code, you will benefit immensely. If anyone is particularly interested in a competition solution, I will provide a detailed analysis later.1) Website Traffic Prediction:
RNN seq2seq model:https://github.com/Arturus/kaggle-web-trafficXGBoost and MLP models:https://github.com/jfpuget/Kaggle/tree/master/WebTrafficPredictionKalman filter:https://github.com/oseiskar/simdkalmanCNN model:https://github.com/sjvasquez/web-traffic-forecasting2) Restaurant Customer Volume Prediction
Feature engineering + LGB:https://www.kaggle.com/plantsgo/solution-public-0-471-private-0-505Feature engineering + LGB:https://www.kaggle.com/pureheart/1st-place-lgb-model-public-0-470-private-0-5023) Open Channel Prediction
Wavenet model:https://www.kaggle.com/vicensgaitan/2-wavenet-swa1D-CNN model:https://www.kaggle.com/kmat2019/u-net-1d-cnn-with-kerasSeq2seq model:https://www.kaggle.com/brandenkmurray/seq2seq-rnn-with-gru4) Lung Pressure Prediction
Transformer model:https://www.kaggle.com/cdeotte/tensorflow-transformer-0-112Bidirectional LSTM model:https://www.kaggle.com/tenffe/finetune-of-tensorflow-bidirectional-lstmTime series problems are vast and have a wide range of applications. In fact, many prediction problems can be seen as time series problems, such as predicting stock/futures/forex prices, website/restaurant/hotel/traffic flow predictions, shop inventory/sales predictions, etc. Mastering time series prediction methods may give you the key to insights into the future.
