Nowcasting the Weather using Machine Learning
- Partners
- KNMI & Utrecht University
- Year
- 2022
Precipitation nowcasting tries to predict the intensity of rainfall in the near future. Due to the dependency of many industries on accurate predictions of nowcasting methods, the development of such methods has increased in recent years. In this research, we validated a Deep Generative Model of Radar (DGMR) developed by Google’s DeepMind on weather data from the Netherlands. The results of the DGMR were compared to a baseline method S-PROG, based on the PySTEPS framework. It was found that the DGMR outperformed the S-PROG method on multiple metrics, scoring significantly higher for Root Mean Squared Error and Critical Success Index after 60 minutes into the prediction. However, the DGMR model often failed to correctly classify predictions at longer lead times. Therefore, it was concluded that this model is capable of making predictions for the Netherlands. For the future, re-training the model will be a next step to achieve the full capabilities of the model.
The challenge
Precipitation nowcasting tries to accurately predict the intensity of rainfall in the near future (e.g., 0-6 hours) at high spatial resolutions. This type of forecasting informs decision-making in many industries. It plays an important role in flood risk assessment due to the possibility of flooding as a result of heavy rain. Therefore, the safety and well-being of many people depend on accurate predictions from nowcasting models. Especially considering the growing frequency and severity of heavy rain events in Europe and the United States.
The development of more accurate precipitation nowcasting models, has increased in recent years. These developments have mostly focused on deep learning approaches because they showed to provide more accurate results than earlier, non-machine learning, methods. Besides, the prediction
of heavy rain events and precipitation, in general, requires the analysis of vast amounts of data which is a task well suited for deep learning approaches. A non-machine learning approach commonly used for nowcasting is the framework PySTEPS which provides many different prediction methods. For example, the Spectral Prognosis (S-PROG) method. This method uses motion fields to predict future radar frames. Motion fields convey the advection of precipitation fields. However, this approach lacks the ability for highly accurate predictions. Mainly due to the addition of a blurring effect, where the predictions get smoothed for increasing lead times. Although this method is able to capture large precipitation structures, it is not able to generate realistic nowcasts. Therefore, blurring is seen as a problem for achieving highly accurate predictions.
In 2021 Google’s DeepMind department launched a new precipitation nowcasting model with a deep learning framework in pursuit of solving the problems of earlier nowcasting models. Here, a Deep Generative Model (DGM) was used to generate predictions based on a series of radar images. This complex model was trained and tested on radar data from the United Kingdom Met Office. Results of this model show it outperforms traditional nowcasting methods, such as S-PROG and STEPS and even perform better than other state-of-the-art machine learning models. Besides, they found that a majority of meteorologists preferred the prediction of this model over those of competing methods after a review of 56 meteorologists from the Met Office. This advanced model could positively influence weather-based decision-making and precipitation nowcasting applications in general, by solving the problems of earlier methods. The results of this deep learning model are promising; however, it is unknown if these results are applicable to areas outside the scope of the original research as of now. This research will further investigate the abilities of this model by validating its performance on Dutch weather data and if it is beneficial to implement this model in the Netherlands. The choice for the Netherlands is well suited for this research due to the availability of high-quality data and its high amount of average yearly rainfall.
Our solution
For this research, we will be validating a deep generative nowcasting model (DGMR) that is introduced and trained by DeepMind. This model predicts N future radar frames based on M past radar frames. These radar frames convey estimates of precipitation intensity. We will explain this model on the basis of the schematic as can be seen in the figure. We see that the input of the mode is split into context and observations, combined these are equal to 22 radar frames of a rain event. Firstly, the context, i.e. the input frames, is equal to four consecutive radar frames (the previous 20 minutes).
The observations, i.e. target frames, are equal to the following 18 frames (the ‘future’ 90 minutes). The latter is only used during training, to adjust the parameters of the model and thereby guide learning.
The 18 predicted frames are then compared to the observed ground truth (i.e. target frames) to compute the Root Mean Squared Error (RMSE), Critical Success Index (CSI) and Fraction Skill Score (FSS) for every timestamp. The average of these measurements over the whole test set will be computed. This procedure is repeated using the nowcasting method SPROG, to provide a baseline. Paired t-tests will be performed to establish significance between the results of both methods, with α = 0.05. Here we will compare the results found at timestamp t0+60 minutes for all metrics. The validation will be run over 100 rain events. This number is chosen because it provides meaningful results without making the runtime unnecessarily long.
The outcome
The results show the DGMR model outperforms the PySTEPS method for all three metrics. This is further emphasized by the results of the paired t-test, showing a significant difference at t0+60 for these two metrics. However, the results from the FSS do not follow the same trend. Furthermore, the metrics show a clear decline off prediction accuracy with increasing lead times. For both RMSE and CSI we also see a bigger difference in prediction accuracy for increasing lead times, in favor of the DGMR model.
We can answer the main research question of this thesis: Can the pre-trained DeepMind precipitation model be used for successful precipitation nowcasting in the Netherlands? This thesis showed that the pre-trained DGMR model introduced by Google DeepMind was able to predict precipitation events for 90 minutes into the future better than the PySTEPS method used as a baseline. Although this is an important step, the predictions still show some room for improvement, especially at longer lead times. However, the intrigues and results of the model in combination with the availability and quality of the data give enough reason to assume that this model, after retraining,
can be used for successful precipitation nowcasting in the Netherlands.
