International Conference on Computer and Knowledge Engineering

Home / 14th International Conference on Computer and Knowledge Engineering

Novel Insights in Deep Learning for Predicting Climate Phenomena

Authors :

Mohammad Naisipour¹ Saghar Ganji² Iraj Saeedpanah³ Behnam Mehrakizadeh⁴ Ahmad Reza Labibzadeh⁵

1- Department of Civil Engineering, Faculty of Engineering, University of Zanjan, Iran. 2- Department of Computer Engineering Faculty of Engineering Shiraz, Iran 3- Department of Civil Engineering Faculty of Engineering Zanjan, Iran 4- Department of Civil Engineering, Faculty of Engineering, Shoushtar Azad University, Iran. 5- Department of Computer Engineering Sharif University of Technology. Tehran, Iran

Keywords :

CNN،Deep Learning،El Niño،Forecast،Evaluation

Abstract :

The rapid advancements in Machine Learning (ML) have demonstrated its potential to uncover novel insights and make predictions of unprecedented accuracy. Nevertheless, before relying on ML in real-world applications, it is crucial to ensure reliability in its results. One of the state-of-the-art machine learning methods for predicting the El Niño-Southern Oscillation (ENSO) phenomenon two years in advance is the Convolutional Neural Network (CNN) published in Nature by Ham et al.‎ [22]. in this study, a comprehensive evaluation of the final forecasts is conducted to gain deeper insight into the deep learning model. We assessed the method's performance using four evaluation metrics with different concepts, which led us to some surprising results. For instance, while correlation and hit rate metrics show a predictability barrier in boreal spring, error metrics show it in austral spring. To ensure a holistic assessment, we also evaluated the method on three distinct periods: the overall period (1984-2017) which was used in the original article, the pre-2000 (1984-2000), and the challenging post-2000 (2001-2017) period. Based on skill-based metrics, the CNN method shows higher forecast skill for the pre-2000 period, whereas, its prediction skill in the new millennia with higher variability is oddly higher according to error-based metrics. This divergence between pre and post-2000 period trends proves CNN's disproportionate skill over different periods. The findings also highlight the contrasting philosophies behind different metrics. The correlation metric targets the ability to detect El Niño or La Nina’s trend without considering its magnitude and strength, whereas value-based metrics like MSE and MAE target the strength and trueness of the results. Thus, to reach a fair judgment about any method, these two kinds of metrics and a holistic assessment from different points of view have to be considered.