Received Date: January 18, 2017; Accepted Date: February 23, 2017; Published Date: February 29, 2017
Citation: Mugume I, Waiswa D, Mesquita MDS, Reuder J, Basalirwa C, et al. (2017) Assessing the Performance of WRF Model in Simulating Rainfall over Western Uganda. J Climatol Weather Forecasting 5:197. doi: 10.4172/2332-2594.1000197
Copyright: © 2017 Mugume I, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Visit for more related articles at Journal of Climatology & Weather Forecasting
Skillful rainfall prediction is important to sectors such as agriculture, health and water resources. The study assessed the ability of the Weather Research and Forecasting model to simulate rainfall over Western Uganda for the period 21st April to 10th May 2013 and tested six cumulus parameterization schemes. The root mean square error, mean error and the sign test method are used to assess the ability of the schemes to simulate rainfall along with an adapted contingency table. Results show that the Grell-Fretas scheme is better at simulating rainfall compared to other schemes over the study period while the Betts-Miller-Janji’c and the Kain-Fritsch schemes overestimated rainfall. However all the schemes under predicted heavy rainfall events but the Betts-Miller-Janjic and the Kain-Fritsch schemes over predicted the light rainfall. The variation of altitude presented a noticeable change in predicted rainfall where an increase of 25% in altitude increased the probability of prediction by 6.5% which shows a key role played by altitude in convection.
WRF model; Adapted contingency; Rainfall; Parameterization schemes
Numerical Weather Prediction (NWP) models, such as the Weather Research and Forecasting (WRF) model have gained widespread attention in weather and climate prediction over the 21st Century. These models are objective  and produce simulations by solving atmospheric governing equations . They have dynamical cores that represent atmospheric processes and physical schemes that resolve the physics in sub-grid scale process. Resolving sub-grid processes requires parameterization such as the cumulus parameterization schemes [3,4] and the microphysical schemes [4,5] which have a great influenced on the precipitation simulated and also play an important role in determining the vertical structure of temperature and moisture fields of the atmosphere .
The use of NWP models in precipitation forecasting is already established in many operational weather and climate prediction centers. This can be partly explained by the demand of improved precipitation prediction since precipitation affects many economic sectors such as agriculture [6,7], fisheries , transport and other economic activities [9,10]. Accurate precipitation monitoring and prediction is thus important for spatial and temporal variability analysis  as well as climate change studies . However, the skill of NWP models regarding quantitative precipitation prediction is a challenge [12,13] which was also observed by Opijah et al.  over the Greater Horn of Africa and is attributed by Ducrocq et al.  to model’s failure to accurately simulate dynamical and physical processes as well as initial conditions. Additionally NWP models normally have systematic biases [1,16] that can be attributed to unresolved sub–grid scale atmospheric processes. In spite of these biases, NWP are used for downscaling coarse resolution atmospheric fields to generate fine scale representation of atmospheric processes [17-19].
Over East Africa, precipitation is majorly in form of rainfall and is influenced by the Inter-Tropical Convergence Zone (ITCZ); the monsoon wind systems; the tropical cyclones; semi-permanent subtropical anticyclones and easterly waves [6,20]. Fortunately NWP models have the ability to simulate these systems as observed by Nishant et al. . Other features that influence rainfall include the complex topography, vegetation and inland water bodies which modulate local rainfall [6,20,22]. The ITCZ passes over the equator twice a year making the region to have two major rainfall seasons, namely, March-May (MAM) and September–November (SON). While passing over East Africa, the ITCZ exhibits both Zonal and meridional flows as discussed by Mugume et al.  but unlike the zonal, the meridional component is shallow (i.e., 2-5 km) .
The amount of precipitation simulated by NWP models is the sum of convective and non-convective rainfall which is contributed by the cumulus scheme and the microphysical scheme respectively. Therefore to assess the ability of WRF model to simulate rainfall over Uganda, it is important to obtain a skilful combination of the cumulus and microphysical schemes  but studies over the tropics such as Mayor and Mesquita  have shown the stronger impact of the cumulus scheme compared to the microphysical schemes regarding convective precipitation events. The different parameterization schemes have biases that depend on region, season or weather but it is important to use a combination with lowest mean error . The cumulus schemes have a couple of assumptions and are based on many variables such as entrainment, cloud height, moist static energy, convective available potential energy and vertical temperature profiles among others.
The study tested six cumulus parameterization schemes by subjecting the simulated results to statistical performance measures including an adapted contingency table. The effect of varying model elevation was also investigated and the study was aimed at answering the questions: (1) What is the most skilful cumulus scheme over Western Uganda? (2) Which scheme is good at simulating light/heavy rainfall events? and (3) What is the effect of varying model elevation on simulated precipitation?
In the study, daily observed rainfall data from 21st April to 10th May 2013 for five stations of western Uganda (Table 1) was obtained from Uganda National Meteorological Authority (UNMA). This data was evaluated using a point-to-point evaluation method with a node interpolated from the prediction grid as explained by Mayor and Mesquita . The model was initialized using boundary conditions obtained from the National Centers for Environmental Prediction (NCEP) final reanalysis  and the resolution of this data is 1° × 1°, covering the period of study (21st April to 10th May 2013).
Table 1: Geographical data of study locations.
The study case
The study was aimed at stimulating rainfall over western Uganda because, during the study period, the region experienced heavy rainfall. For example in Masindi, 124.9 mm of rainfall was recorded on 6th May 2013. Over Kasese, heavy rainfall of 40.1 mm pounded the region on 1st May 2013 and caused devastating flooding in Kasese district after bursting of the river banks of Nyamwamba and Mobuku. According to the Disaster Relief Emergency Fund of the International Federation of Red Cross and Red Crescent, eight people were confirmed dead and an estimated 9,663 people were displaced as a consequence of the floods. In addition, infrastructure such as houses and bridges were destroyed.
The study was conducted using the WRF version 3.8 model which consist of a dynamical core and physical parameterization schemes and it is recommended for high resolution simulations by Mayor and Mesquita . This model is developed for regional simulation and weather prediction  and is popularly employed in NWP simulations by many scholars including Mercader et al. , Flaounas et al. , Krogsater et al. , Rajasekhar et al.  and many others.
For our study, the mercator map projection was used for projecting nested domain (Figure 1) with the parent domain extending from 18.3306°N to 23.0638°S and 0.42832°W and 55.4283°E at a horizontal resolution of 30 km while the nest domain extended from 6.04777°N to 5.1772°S and 25.9259°E to 39.8676°E at a horizontal resolution of 10 km.
The coarse domain was used for large scale simulation while the nest for validation of the schemes using station gauge observed rainfall data. The model top was set at 50 hPa with 40 vertical layers.
The study was carried out to assess the performance of cumulus parameterization schemes because they are vital to the model’s ability to simulate precipitation and therefore needs accurate representation . Thus the other physical parameterization schemes were set to default and only the cumulus schemes varied. A couple of cumulus schemes have been developed based on different closure assumptions [30-34].
For the study we used six cumulus schemes namely the Kain– Fritsch (KF), the Betts–Miller–Janji´c (BMJ), the Grell–Fretas (GF), the Grell 3D ensemble (G3), the New–Tiedke (NT) and the Grell– Devenyi (GD) (Table 2) and we allowed a spin–up period of 24 hours. For all the experiments, we used WRF Single Moment 3–class scheme as the microphysical scheme and the Yonsei University scheme as the planetary boundary layer scheme.
|KF||The KF is a mass flux scheme that uses Langragian parcel method and is used at NOAA/NWS/SPC|
|BMJ||The BMJ is a moist column adjustment scheme relaxing towards a mixed profile but without explicit updrafts or downdrafts and no cloud detrainment |
|GF||The GF is an improved GD scheme that smoothens the transition to cloud– resolving scales and has explicit updrafts and downdrafts including cloud ice and detrainment |
|G3||Is an improved scheme of GD which is a one-dimensional mass flux scheme having a combination of multiple closures based on convective available potential energy, moisture convergence and allows subsidence to occur within the same grid column or spreading to neighboring grids|
|NT||The NT is a mass–flux scheme with CAPE removing time scale and also having a shallow component and momentum transport|
|GD||The GD is a multi–closure, multi–parameter ensemble with different updraft and downdraft entrainment and detrainment parameters|
Table 2: Description of cumulus schemes.
Evaluation of the cumulus schemes in different areas of the world has been carried out and results show varying performance of the different cumulus schemes. For example, Ratna et al.  evaluated KF, BMJ and GDE over South Africa and found all the schemes to produce positive biases and KF produced the largest biases and largest mean absolute error.
In a separate study of impacts of physical parameterization schemes on seasonal rainfall, Jankov et al.  noted that BMJ normally over estimated light rainfall events.
In the study, we used both parametric and non-parametric methods to comprehensively assess the performance of cumulus schemes. The parametric methods consider rainfall as a continuous random variable both in space and time and are based on difference between simulated and observed rainfall. For the study we used the Root Mean Square Error (RMSE); and mean error (ME or bias); as parametric methods which are popular statistical scores for assessing the skill of NWP models as observed by Mercader et al.  and the Sign Test Method (STM) as a non-parametric method as described by Mugume et al.  due to its insensitivity to outliers.
Model difference: For paired data-sets (i.e., simulated, M and observed, O), let the difference between simulated and observed data be d such that:
Where l is the cumulus scheme and k is kth data point ordered in time. A set of differences is thus obtained as:
Where n is the total number of data points ordered in time.
Root mean square error: The Root Mean Square Error (RMSE) is the square root of the mean square difference and is computed mathematically as:
The model altitude was varied and the changes in RMSE (i.e., ΔRMSE) presented in a Table (8) below.
Mean error: The mean error (ME), also known as bias is the mean of the model differences and is computed as:
The model altitude was varied and the changes in ME (i.e., ΔME) also presented in a Table (8) below.
Sign test method: The STM is described by considering a statistic, v:
the values of , i.e., are averaged to give STM for a given scheme (STMl), i.e.
The model altitude was varied and the changes in STM (i.e., ΔSTM) also presented in a Table (8) below.
The categorical method: We also used the categorical methods that discretize rainfall. The most common categorical tool is the 2 × 2 contingency table (Table 3) which considers rainy events in terms of occurrence (i.e., Yes/No) as explained by Done and Davis . The problem of such a 2 × 2 contingency table when used for assessing the performance of an NWP model’s ability to predict rainfall is that it treats all rainfall events the same and thus fails to capture the possible bias of the model. To attempt to address this limitation, we suggested and adopted a ’5 × 5 contingency table’ with different bins (i.e., rainfall=0, rainfall <10 mm, rainfall <20 mm, rainfall <30 mm and rainfall ≥30 mm). Although we used a ‘5 × 5 contingency table’, this table can be adopted to have different ranges. We give a comprehensive account of how the contingency table can be adapted to extended ranges i.e., an ‘n × n contingency Table’ (Table 4).
|No||False Alarm||Correct No|
Table 3: The 2 × 2 contingency table.
Table 4: The n × n contingency table.
For n bins including a special case of ‘no rain’ (i.e., rainfall=0), we can construct an n × n square matrix obtained from an n × n contingency table. Suppose that n columns represent the simulated results and n rows represent the observed results and let i represent the ith column and j represent the jth row. The aij’s are the number of cases falling in the ith simulated and jth observed category. We thus extract our performance matrix (M) as:
Since the columns correspond to simulated values and the rows correspond to observed values, the performance matrix can be decomposed into ‘Upper triangular matrix’ (UM), ‘Diagonal matrix’ (DM) and ‘Lower triangular matrix’ (LM).
We defined three performance measures thus, ‘Probability of over Prediction’ (POV), ‘Probability of Prediction’ (POP) and ‘Probability of Under Prediction’ (PUN). The POV is the average of values giving the Upper Triangular Matrix (UM) i.e., < aij > for all and strictly i > j values, POP is the average of value making the Diagonal Matrix (DM) i.e., < aij > for all i=j and PUN is the average of value making the Lower Triangular Matrix (LM) i.e., < aij > for all i<j. Thus if N is the number of cases over which performance is being measured, we can calculate POV, POP and PUN using equations (6 to 8).
An additional advantage of our proposed n × n contingency table is that it can help in determining the performance of a model in a given range of rainfall. As an illustration given a range e.g. obsk we can investigate aik such that for all i>k values correspond to over prediction; for all i<k values correspond to under prediction and i=k is value simulated is in the range. In general we suggest that a good model (e.g. a cumulus scheme for our study) should have a higher POP compared to the rest and a smaller POV+PUN. A higher POV indicates a higher probability of over prediction in the same way a high PUN indicates a higher probability of under prediction.
The performance of cumulus schemes
The Tables 5 and 6 as well as (Figure 2) represent the performance of cumulus schemes. These results are obtained by comparing grid–point simulated rainfall, averaged over a point with corresponding observed rain gauge rainfall. Table 5 presents results obtained using a 5 × 5 contingency table described in analysis method while Table 6 presents statistical results (i.e., RMSE, ME and STM). Our results show that the GF scheme had the highest probability of giving rain falling in the same range as observed (i.e., POP) while NT had the smallest POP. The POP of the GF is marginally greater than that of BMJ (i.e., GF: 46% & BMJ: 45%). The drawback of BMJ is that it has a higher PUN compared to GF (i.e., BMJ: 40% & GF: 25%) which makes the GF a better cumulus scheme.
|KF (%)||BMJ (%)||GF (%)||G3 (%)||NT (%)||GD (%)|
Table 5: Results from a 5 × 5 contingency table for performance of cumulus schemes. The measures are in percentages.
Table 6: Statistical Indices showing the RMSE, ME and STM of the cumulus schemes investigated.
Additional investigation using statistical indices (i.e., using the RMSE, ME and STM) as presented in Table 6 revealed that the NT cumulus scheme had the smallest RMSE (i.e., RMSE=12.71) and again the BMJ scheme had a marginally greater RMSE compared to GF (i.e., BMJ: 13.47 & GF: 13.31). Although the GD scheme had the smallest magnitude of ME (i.e., ME=-0.26), unfortunately it also had the highest magnitude of RMSE (i.e., RMSE=19.12). This GD scheme also had a high POP (42%) and a small STM (-0.01). Our problem with GD scheme is the high variability of the statistical indices (Table 6) over different stations.
We noted that the ME over individual stations are big but are compensated by averaging over the study region to appear to give a small ME. Overall, we find the GF scheme superior due to a fairly low RMSE (13.31), ME (1.17) and STM (-0.02).
Additional analysis of accumulated rainfall time series (Figure 2) generally revealed that the BMJ & KF schemes simulating excess accumulated rainfall with exception of Kasese station (Figure 2c). Over Masindi station (Figure 2e) all the cumulus schemes under–predicted but the KF scheme fairly simulated the rainfall over the period but like other schemes considered, it failed to simulate the heavy rainfall that occurred on 6th May 2016 (i.e., total rainfall of 124.9 mm).
Light and heavy rainfall events
The results for simulated heavy rainfall events and light rainfall events over the study period are presented in Table 7. A heavy rainfall day is one with total rainfall amount greater than 20 mm while a light rainfall day is one which total rainfall amount less than 1 mm . We noted that all the schemes under–predicted the heavy rainfall events over the period of study and that the KF scheme had fairly the smallest magnitude of error compared to other schemes. This means that the KF scheme is better at simulating heavy rainfall events. For light rainfall events, we noted that KF scheme over–predicted rainfall amount (i.e., RMSE: 8.22; ME: 3.54 & STM: 0.21) followed by BMJ scheme (i.e., RMSE: 7.19; ME: 3.04 & STM: 0.38). These results confirm that KF and BMJ schemes over–predicted rainfall over the study period and are in line with a similar study by Ratna et al.  that showed that the KF scheme had a more unstable convective environment compared to the GD scheme thus over predicting convective rain compared to stratiform rain. Considering the GF scheme, we noted that it presented a higher mean error regarding heavy rainfall which probably indicates that the GF scheme is not good at simulating heavy rainfall events .
|Schemes||Heavy rainfall events||Light rainfall events|
Table 7: Statistical results for simulation of heavy and light rainfall events over the study period.
Effect of altitude on simulated precipitation
The analysis of altitude (Table 1) and simulated precipitation showed that, Kabale which had the highest altitude (1,869 m) also had 4/6 schemes (i.e., BMJ, GF, G3 & NT) with the highest RMSE over it. Additional analysis of Figure 2b), for Kabale show that all the schemes over estimated rainfall. For Mbarara, which has altitude 1,420 m, all the schemes also over predicted (Figure 2d). Kasese and Masindi which were lower in altitude compared to the rest (i.e., 691 m and 1,147 m respectively) also had the schemes under predicting rainfall (Figure 2c and 2e respectively). Other studies e.g. Maussion et al.  found WRF model to over predict rainfall over high altitude and complex topography and called it orographic bias. Our results probably also confirm an orographic bias but the only problem is lack of validation data over high altitude areas like mountain Rwenzori.
To further investigate the effect of altitude on simulated precipitation, the altitude was decreased by 25%, 50% and also increased by 25% and 50%. The results for statistical indices (RMSE, ME & STM) are presented in Table 8; contingency results presented in Table 9 and time series for accumulated rainfall in (Figure 3). Our results do not necessary show improvement in simulation of precipitation but reveal changes in the amount of precipitation simulated (Table 8).
Figure 3: The Figures 3a-3e shows the time–series of cumulated rainfall for the stations used in the study. Each figure has respective graph of actual observations (obs), the simulated rainfall with model altitude (GF-0) and the simulated rainfall with varying altitudes i.e., reduced by 25% (GF 25), reduced by 50% (GF 50), increased by 25% (GF.25) and increased by 50% (GF.50)
|RMSE for varied altitude||Change in RMSE (%)|
|ME for varied Altitude||Change in ME (%)|
|STM for varied Altitude||Change in STM (%)|
Table 8: The RMSE, ME and STM for various altitude considered of GF cumulus schemes. GF-orig is the original GF scheme with default altitude; GF-25 is GF scheme with altitude reduced by 25%; GF-50 is GF scheme with altitude reduced by 50%; GF+25 is GF scheme with altitude increased by 25% and GF+50 is the GF scheme with altitude increased by 50%.
|GF-0 (%)||GF-25 (%)||GF-50 (%)||GF+25 (%)||GF+50(%)|
Table 9: Contingency table for performance of cumulus schemes with varied altitude.
We noted that when we varied altitude, there was a noticeable change in RMSE, ME and STM. Different stations experienced different changes and this could probably be attributed to changes in centers of formation of precipitating clouds since convective clouds are influenced by both diabetic heating, orography and moisture advection . The largest positive changes in both ME and STM are observed when altitude is reduced by 50% whereas increase of altitude by 50% gave negative changes.
The Figure 3 also show that reduction of altitude by 50% resulted to over prediction of rainfall changing the bias from negative to positive especially for Mbarara and Kabale. These results are also confirmed by the changes in total rainfall over the study period due to variation in altitude Figure 4 but we noted that although reduction of altitude by 50% generally increased total rainfall, it was still under estimate for Bushenyi, Kasese and Masindi which could be due to the weaker ability of the GF scheme to simulate heavy rainfall events.
Figure 4: The bar chart showing changes in amount of total rainfall over the study period due to variation in altitude. GF+50 means that altitude was increased by 50%; GF+25 means that altitude was increased by 25%; GF–50 means that altitude was reduced by 50%; GF–25 means that altitude was reduced by 25%; GF–0 means that model altitude was maintained and obs is actual rainfall observation.
In terms of predictive ability, the results of extended contingency table (Table 9) shows that the predictive capacity of the GF scheme improved from 46% to 49% by increasing the altitude by 25%.
As already observed in Table 8 that the increase of altitude by 50% resulted to increased underestimate of rainfall, we also see in Table 9 that predictive ability declines to 40% and Probability of Underestimation (PUN) increases from 25% to 35%. Thus elevation can influence when and where precipitation happens .
The study assessed the performance of WRF model regarding rainfall simulation over Western Uganda by investigating six cumulus schemes, namely the Kain–Fritsch, the Betts–Miller–Janji’c, the Grell– Fretas, the Grell 3D ensemble, the New–Tiedke and the Grell–Devenyi using statistical indices (i.e., RMSE, ME and STM) as well as a ‘5 × 5 extended contingency table.
Additionally, we investigated the ability of the schemes to simulate heavy rainfall events (i.e., a heavy rainfall day is one with total rainfall amount greater than 20 mm) and light rainfall events (i.e., a light rainfall day is one with total rainfall amount less than 10 mm) over the study period.
We found that the GF scheme had the highest Probability of Predicting (POP) rainfall while NT had the smallest POP. We also noted that although the GD scheme had the smallest magnitude of ME (i.e., ME=-0.26), it had the highest magnitude of RMSE (i.e., RMSE=19.12) and its other drawback was the high variability of the statistical indices over different areas which are compensated by averaging over the study region to appear to give a small ME. Generally we found the GF scheme a superior scheme due to a fairly low RMSE (13.31), ME (1.17) and STM (-0.02).
We also observed that all the schemes under–predicted heavy rainfall and that the KF scheme had fairly the smallest magnitude of error compared to other schemes which meant that the KF scheme is better at simulating heavy rainfall events. For the light rainfall events, we found the KF scheme to over–predict followed by BMJ scheme which confirmed the KF and BMJ schemes to over–predict rainfall over the study period and because the GF scheme presented a higher mean error regarding heavy rainfall, it probably indicated that it is not good at simulating heavy rainfall events.
Further, we found that the altitude had an influence on simulated precipitation and found that, Kabale which had the highest altitude also had 4/6 schemes (i.e., BMJ, GF, G3 & NT) with the highest RMSE and all the schemes over estimated rainfall over it. Kasese and Masindi which had a lower altitude compared to the rest also experienced under–prediction of rainfall which confirms an orographic bias. When the altitude was varied, the results did not necessary show improvement in precipitation simulation but revealed changes in the amount of precipitation simulated. We noted a change in RMSE, ME and STM which could probably be attributed to changes in centers of formation of precipitating clouds since convective clouds are influenced by many factors including orography. The largest positive changes in both ME and STM were observed when altitude was reduced by 50% whereas increase of altitude by 50% gave negative changes. The predictive ability of the GF scheme improved from 46% to 49% by increasing the altitude by 25% which emphasize the influence of elevation on convective rainfall
In the study we also adapted a ‘2 × 2 contingency table’ to a 5 × 5 contingency table that enabled us to fairly assess whether the model over predicts, predicts rainfall in the a given range and or under– predicts. This adapted contingency table can also assist in assessing the ability of a model to predict rainfall in a given range (i.e., rainfall greater than a given threshold e.g. greater than 20 mm). Finally, we recommend the GF cumulus scheme as a basic deterministic scheme which can also be used as a convection scheme to give a background analysis during data assimilation but for heavy rainfall events, we recommend the BMJ scheme.
The authors are grateful to the project of “Improving Weather Information Management in East Africa for effective service provision through the application of suitable ICTs” (WIMEA-ICT project: UGA– 13/0018) under the Norad’s Programme for Capacity Development in Higher Education and Research for Development (NORHED) for technical support and the project of “Partnership for Building Resilient Ecosystems and Livelihoods to Climate Change and Disaster Risks” (BREAD project: SIDA/331) under the “Swedish International Development cooperation Agency” (SIDA) for the financial support and to the reviewers for the constructive feedback. We are also grateful to UNMA (https://www.unma.go.ug) for availing the rainfall data used in the study.