Long-Term ENSO Forecasting Using Hybrid CNN and Transformer Models

Lyu, P. M., T. Tang, F. H. Ling, J.-J. Luo, N. Boers, W. L. Ouyang, and L. Bai, 2024: ResoNet: Robust and Explainable ENSO Forecasts with Hybrid Convolution and Transformer Networks. Adv. Atmos. Sci., https://doi.org/10.1007/s00376-024-3316-6

http://www.iapjournals.ac.cn/aas/en/article/doi/10.1007/s00376-024-3316-6

In recent years, machine learning methods have been increasingly applied in the field of climate prediction analysis. Existing research indicates that neural network models can extend the effective prediction time of El Niño-Southern Oscillation (ENSO) events to over one and a half years. However, there are still some doubts regarding the robustness of the predictions made by neural network models. Moreover, due to the complex structure of these models, the scientific validity and interpretability of their predictions still need to be explored. To address this, this paper proposes the ResoNet model, which combines Convolutional Neural Networks (CNN) and Transformer architectures. CNNs can capture local sea temperature anomalies, while Transformers are adept at extracting information and analyzing overall phase changes. The combination allows ResoNet to fully describe the sea temperature characteristics related to ENSO. Experimental results show that ResoNet outperforms existing methods in predicting the ENSO index over a 19-month forecasting time scale. By applying interpretability algorithms, the prediction logic of ResoNet for forecasting El Niño and La Niña events from 1 to 18 months was further analyzed, revealing that the prediction logic of ResoNet aligns with various physical mechanisms, such as the Pacific recharge-discharge mechanism, the Indian Ocean capacitor mechanism, and the seasonal footprint mechanism. Additionally, through heat map analysis, this paper finds that ResoNet can capture the asymmetry in the evolution of El Niño and La Niña. The findings of this study are expected to reduce skepticism towards the application of machine learning methods for ENSO predictions and encourage more attempts to utilize artificial intelligence methods to discover and predict climate phenomena.

Keywords: machine learning, El Niño-Southern Oscillation, Convolutional Neural Networks, Transformer

This paper adopts neural network algorithms, effectively integrating Convolutional Neural Networks and Transformers to propose a new network framework, ResoNet, which extends the effective prediction duration of ENSO to over 19 months, enhancing the effectiveness and robustness of El Niño and La Niña (ENSO) predictions. By utilizing artificial intelligence interpretability algorithms such as Integrated Gradients, this paper analyzes the prediction logic of ResoNet for ENSO, finding that its physical mechanisms align with many existing models, such as the recharge-discharge model and seasonal footprint mechanism, while also automatically discovering asymmetries in the formation processes of El Niño and La Niña. This paper also finds that the Bagging Algorithm can improve the robustness of neural network predictions and prevent overfitting.

Special Issue on “AI Applications in Atmospheric and Oceanic Science: Pioneering the Future”

http://www.iapjournals.ac.cn/aas/en/virtualTopic?id=9134e796-acef-402e-8c28-dcd05c6816d3