PyTorch-Forecasting[1] makes time series forecasting using neural networks simple for data scientists and researchers.
Why Accurate Forecasting is So Important?
Forecasting time series is crucial in many scenarios and is highly relevant to machine learning practitioners. Take demand forecasting as an example, many use cases stem from this. Almost every manufacturer benefits from a better understanding of their product demand to optimize production quantities. Underproduction leads to lost revenue, while overproduction forces you to sell excess products at a discount. Pricing is also highly relevant, as it is essentially a form of demand forecasting that focuses specifically on price elasticity. Pricing concerns nearly all companies.
For a multitude of additional machine learning applications, time is critical: predictive maintenance, risk scoring, fraud detection, etc. The order of events and the time between them are vital for creating a reliable forecast.
In fact, while time series forecasting may not be as glamorous as image recognition or language processing, it is more common in industry. This is because image recognition and language processing are relatively new fields often used to power new products, while forecasting has been around for decades and is central to many decision-support systems. Adopting high-precision machine learning models, such as those in PyTorch Forecasting, can better support decision-making and even automate it, often directly leading to millions of dollars in additional profits.
Deep Learning Emerges as a Powerful Forecasting Tool
Deep learning (neural networks) has surpassed traditional methods in time series forecasting in recent years and performs better than in image and language processing. In fact, deep learning only surpassed traditional statistical methods in forecasting pure time series (meaning without covariates, e.g., price is a function of demand) two years ago. However, with the rapid development of the field, the accuracy advantages associated with neural networks have become very apparent, making it worthwhile to increase their use in time series forecasting. For example, the latest architecture N-BEATS[2] has shown an 11% decrease in sMAPE compared to the second-best non-neural network methods (i.e., a collection of statistical methods) on the M4 competition dataset. This network is also implemented in PyTorch Forecasting[3].
Moreover, even compared to other popular machine learning algorithms like gradient boosting trees, deep learning has two advantages. First, the design of neural network architectures can inherently understand time, meaning they automatically establish connections between data points that are close in time. Thus, they can capture complex temporal dependencies. In contrast, traditional machine learning models require manual creation of time series features, such as the average of the past x days. This undermines the ability of these traditional machine learning algorithms to model temporal dependencies. Second, most tree-based models output a step function by design. Therefore, they cannot predict the marginal impact of changes in input, and are notoriously unreliable in out-of-domain predictions. For instance, if we only observe prices of €30 and €50, a tree-based model cannot assess the impact of a price change from €30 to €35 on demand. Thus, they often cannot be directly used for optimizing inputs. However, that is often the whole point of creating machine learning models—their value lies in optimizing covariates. At the same time, neural networks use continuous activation functions, making them particularly adept at interpolating in high-dimensional spaces, meaning they can be used to optimize inputs like price.
What is PyTorch Forecasting?
PyTorch Forecasting aims to simplify time series forecasting for real-world cases and research using neural networks. It provides state-of-the-art time series forecasting architectures that can easily be trained with pandas data.
- The high-level API significantly reduces the user’s workload, as users do not need specific knowledge of how to prepare training datasets using PyTorch. The TimeSeriesDataSet class handles issues such as variable transformations, missing values, random subsampling, and variable lengths. You only need to provide a pandas DataFrame and specify which variables the model should learn from.
- The BaseModel class provides generic visualization capabilities, such as displaying predictions against actuals and partial dependence plots. Metrics and training progress can be automatically logged in tensorboard[4].
- State-of-the-art networks are used for predictions with and without covariates. They also come with built-in interpretability capabilities. For example, temporal fusion transformers[5][3] have outperformed Amazon’s DeepAR in benchmarks by 36-69%, with variable and time importance measures. More on this can be seen in the examples below.
- There are many multi-horizon time series metrics available to evaluate prediction results across multiple forecasting horizons.
- To achieve scalability, these networks are designed to work with PyTorch Lightning[6], allowing for out-of-the-box training on CPU and single or multiple (distributed) GPUs. The implementation of the Ranger optimizer is aimed at faster training of models.
- For ease of experimentation and research, adding networks is straightforward. The code is designed with PyTorch experts in mind. They will find that even complex ideas can be easily implemented. In fact, one only needs to inherit the BaseModel class and follow the conventions of input and output for the forward method to immediately enable logging and interpretability features.
To get started, detailed tutorials in the documentation demonstrate the end-to-end workflow. I will also discuss a specific example later in this article.
Why Do We Need This Package?
PyTorch Forecasting helps overcome significant barriers to using deep learning. While deep learning has dominated in image and language processing, it has not been the case in time series forecasting. The field is still dominated by traditional statistical methods (like ARIMA) and machine learning algorithms (like gradient boosting), with Bayesian models being the only exception. There are two main reasons why deep learning has not become mainstream in time series forecasting, both of which can now be overcome:
- Training neural networks almost always requires GPUs, which are not always readily available. Hardware requirements are often a significant barrier. However, this obstacle can be overcome by moving computations to the cloud.
- Using neural networks is relatively more difficult compared to traditional methods. This is particularly true for time series forecasting. There is currently a lack of high-level APIs that work with popular frameworks like Facebook’s PyTorch or Google’s TensorFlow. For traditional machine learning, there exists the sci-kit learn ecosystem, which provides practitioners with a standardized interface.
Given its user-unfriendliness requiring extensive software engineering, this third barrier is considered crucial in the deep learning community.
Editor / Zhang Zhihong
Review / Fan Ruiqiang
Verification / Zhang Zhihong
Click below
Follow us
Read the original text