Keywords

Introduction

Background

Weather and climate affect man in diverse ways which greatly impact agriculture, food security, housing, transportation, health, engineering structures, water resources, military operations, oil exploration and exploitation, environment, livelihoods, etc. (Ugbah 2016). These impacts are usually associated with extreme weather and seasonal hazards such as tropical cyclones, heavy rainfall, flooding, etc., which occur more frequently (IPCC 2018). For these reasons, scientists have continued to invest great time and resources in trying to understand and predict the processes that lead to changes in the atmosphere associated with these hazards. One of such investment is the international project known as the subseasonal to seasonal (S2S) prediction project set up in 2013 by the World Meteorological Organization (WMO), World Weather Research program (WRWP), and World Research and Climate program (WRCP) to provide a common platform for sharing of data and knowledge between countries, National meteorological Agencies, and researchers which will enhance our understanding of seasonal and subseasonal variability and also help to improve the skill and accuracy of long range forecast (weeks–months) and likely extreme events in different parts of the world (Takaya 2015).

In most parts of Asia and Africa and in developing countries such as Nigeria, farmers still practice subsistence farming which relies mainly on seasonal rainfall for Agricultural production (Siegmund et al. 2015; Hansen 2002). Farming, out of all human activities, is the most weather-dependent (Hanson 2002). A timely and accurate prediction of the onset, cessation, expected rainfall of the monsoon season, and its intra-seasonal variability is very likely to reduce losses and risk of extreme weather (Vitart et al. 2016) as well as maximize agricultural output to ensure food security. This is one of the purposes for which the S2S project was set up.

The time range for subseasonal forecast lies between 2 weeks and 60 days (Vitart et al. 2016) within the rainy season and it is meant to “bridge the gap between the medium range forecast” (at most 2 weeks) and seasonal forecast (3–6 months) [Vitart et al. 2016; Molteni et al. 2016]. The Madden Julian Oscillation (MJO), among other factors such as El-Nino Southern Oscillation (ENSO), stratospheric influence, Ice and snow cover, soil moisture, ocean conditions, and tele-connection between the tropics and extra tropics, is said to be the main source of predictability for subseasonal time range (Vitart et al. 2016; Inness and Dorling 2010); this may not be unconnected with the fact that MJO plays a significant role in initiating or terminating equatorially trapped waves (Kelvin waves) and Rossby waves (Maclachlan et al. 2015) which influence ENSO phenomena. The choice of the ECMWF forecast rather than others is based on the high skill displayed by the model in simulating MJO over the tropics and extra tropics, and the continuous improvement in skill of predicting subseasonal variability and events up to lead time of 10 days, since 2002. This is attributed to the improvement in the ECMWF model characteristics (resolution, initial conditions, physics, etc.) made possible because of the advancement in information technology which has led to the production of better computers with higher computational capabilities, capacity, and speeds in recent times. This development gives a ray of hope to those who depend on subseasonal to seasonal rainfall for production, planning, decision making, disaster preparedness, and socioeconomic development, thus the need to embrace the S2S project.

This study was done in 2016 with the aim of assessing the skill of ECMWF subseasonal ensemble forecasting system in simulating subseasonal rainfall variability in terms of pattern and amount of rainfall in Nigeria up to lead time of one month specifically over southern Nigeria where annual rainfall is highest and the risk of flooding from excessive rainfall also high. The area is highly threatened by sea level rise especially during the rainy season. This work also investigates how good the S2S model ensemble is, in predicting the likelihood of extreme rainfall that may cause flooding over the region.

The scope of this work is limited to southern Nigeria only where risk of flooding is highest and does not cover the entire country. Also it does not include evaluation of how the model can simulate or predict rainfall onset, cessation, or length of season but rainfall amount and pattern. It is expected that findings from this research will not only help to improve the quality and accuracy of S2S forecast, but will also help in updating the seasonal forecast of Nigerian Meteorological Agency and provide useful information and early warning system for disaster preparedness and risk reduction against extreme weather and climate events such as flooding.

Review of ECMWF Subseasonal to Seasonal Forecasting

In 2002, Frederic Vitart started the ECMWF medium range (monthly) forecast of precipitation as an experimental phase which became fully operational by 2004. It was integrated with the ensemble prediction system in 2008 following results of improved skill in simulation of weekly surface temperature, precipitation, mean sea level, and MJO observed during the period over the extra-tropics (Vitart 2005, 2014; Vitart et al. 2016). The model is run two times in a week to produce a daily forecast in real time for lead period up to 32 days, with a 1 hourly high resolution retrieval, and a reforecast (hindcast) for more than 12 years (Molteni et al. 2016; Vitart et al. 2016). The forecast consists of 51 ensemble members while the reforecast has 11. According to Vitart et al. (2016), it is structured to allow better coupling of the atmosphere (IFS model) and ocean (NEMO model) compared to models used in NOAA and UKMet office. Representation of model physics and physical processes has also been improved in very recent version. Some of the improvements include the increase in frequency of the S2S forecast from every 14 days in 2002 to twice a week from 2013 to date. The horizontal resolution has also evolved from T159 which had a lead time up to 32 days in 2002, through T255, T319, and presently T639 with lead time of up to 46 days. Vertical resolution has also improved from 40 atmospheric levels with top at 10 hPa in earlier versions to 91 atmospheric levels with top at 1 Pa in 2016. Re-forecast size now has 11 ensemble members as against 5 members in earlier versions (Molteni et al. 2016).

Work by Vitart (2005; Vitart et al. 2016) in trying to evaluate the skill of ECMWF subseasonal forecast showed higher weekly scores for lead times greater than 10 days and better skills when compared to persistent forecast and climatology (Vitart 2005; Vitart et al. 2016; Haiden et al. 2015). In another work, verification of the ECMWF subseasonal forecast model was done using metrics such as anomaly coefficient of correlation (ACC) and root mean square error RMSE and probabilistic skills (Vitart 2004). This approach is very popular and it is in line with the WMO recommendation on verification of deterministic forecast (Inness and Dorling 2010). Vitart’s result showed decreased correlation in the weekly anomalies with increasing lead time; the highest correlation value was observed in the first week forecast. The skill was found to be greater than skill of the persistence forecast. For the ensemble probability forecast, the relative operative characteristics (ROC) scores which show the ratio between the Hit rate and false alarm were determined using the contingency table. ROC value of 0.5 means “no skill forecast” while values above 0.5 suggest good skill of forecast. The ROC values obtained from Vitarts work showed that forecast of 12–18 days was better than persistence forecast of 5–11 days. His work in 2014 also confirms this, with highest skill for week 1 (0.8) decreasing to 0.6 during week 3 (Vitart 2014). Brier score of ECMWF subseasonal forecast was 0.04 which suggested that model performed better than climatology (Vitart et al. 2016). The result of evaluation of the ECMWF subseasonal forecast showed better performance than other global centers as seen from the studies of Vitart over the extra tropics and consistent with other centers (Haiden et al. 2014). He pointed out that there could be variability in scores from one region to another and so, it would be wrong to conclude that the African region with a different climate, weather and geomorphology, will produce same skill as the extra-tropics without carrying out studies to compare and make valid conclusions, hence the justification for this study over Nigeria.

Li and Robertson (2015) also carried out studies to evaluate precipitation forecast from global ensemble prediction system at submonthly timescales using hind cast data from three global prediction centers (ECMWF, NCEP, and JMA) during May to September for the period 1992–2008. Statistical metrics used were the Correlation of Anomalies (CORA), and the Mean Square Skill Score (MSSS). The results showed good skills from all the three models during the first week compared to lead times of 2–4 weeks. The skill from ECMWF forecast was however better than the other two. Anomaly correlation skill scores were found to be high at lead times 1–4 over the equatorial Pacific while values of 0.2–0.3 were found to be statistically significant over tropical Atlantic at lead 1 only. The skills over land areas were generally poor even at lead time of one week over Africa. The result also showed very good correlation between ENSO and the submonthly prediction with ACC skills for ENSO years higher than neutral years. Biases in mean weekly precipitation forecast were large particularly in the NCEP model over the Atlantic, West Africa, and the Sahel regions.

Studies by Hamill (2012) on verification of the ECMWF probabilistic precipitation forecast, TIGGE National Centers for Environmental Prediction (NCEP) model, Canadian model, and UK Met Office (UKMO) forecast models during June–November 2002 to 2009 over the United states of America showed that ECMWF had the best skills with better scores for higher precipitation. These works confirm the leading standard of the ECMWF model forecast which has been chosen for this study.

Description of the Study Areas and Methodology

Study Area

Nigeria is located towards the coast of West Africa on latitude 4.0oN–14.0oN and longitude 2.45°E–15.5°E (Fig. 1). It occupies a landmass of about 924,000 Km2 and population of about 133 million as at 2006 which grows at a rate of 2.8% (Ayanlade et al. 2013). The study area is the southern part of the country shown in the red box (Fig. 1) defined by latitude 4.0oN–8.0oN and longitude 2.0°E–11.0°E estimated from the figure which is sourced from Eludoyin and Adelekan (2013). It covers an area less than 50% of the country and is bounded in the South by the Atlantic Ocean (Gulf of Guinea). The area is bounded in the East and West by two countries: Cameroun and Benin Republic, respectively.

Fig. 1
figure 1

Map of Nigeria showing study area in rectangular box (Source: Eludoyin and Adelekan 2013, springer publishers)

Full size image

Climatology and Sources of Subseasonal Predictability in the Region

The study area lies within the mangrove swamp, tropical rainforest, and Guinea Savanna vegetative zones which experience two major seasons: Wet (late February to November) and the dry (December to mid-February) as reported by Eludoyin and Adelekan (2013). The unequal distribution of rainfall from month to month within these two major seasons further distinguishes the season into long rainy season (March–July), short dry season or August break (Late July–early August), short rainy season (early September–mid-October), and the long dry season (late October–early March) (Chidiezie and Shrikant 2010). The wet season is said to have a bimodal pattern of rainfall and a longer period which lasts about 200–300 days (NiMet 2013, 2014). It is characterized by moist southwesterly winds also known as the tropical maritime air mass (mT) which blow landwards from the Atlantic Ocean and is predominant during the rainy season. The dominant winds during the short period of dry season (Harmattan season) are the dry dusty northeasterly trade winds from the desert also called the tropical continental air mass (cT) which find their way to the South from the North (Eludoyin & Adelekan 2013). Annual rainfall total ranges from 1500 to 3000 mm over most of the South but can reach about 4000 mm over the southeastern part where rainfall is highest (Ayanlade et al. 2013, Chidiezie and Shrikant 2010). Mean air temperatures are in the range of 25.0°–30.0 °C during summer and 20.0–30.0 °C during the harmattan season (Ayanlade et al. 2013)

Research Methods

Study Site Selection and Sampling Methods

The study area covers all locations in southern Nigeria which lie between latitude 4.0 and 8.0 N and longitude 2.0 and 11.0 E which are Bayelsa, Rivers, Akwa-Ibom, Cross-river, Ebonyi, Enugu, Abia, Anambra, Imo, Delta, and Edo states. Others are Ondo, Lagos, Ogun and southern part of Osun, Oyo, Benue, and Kogi states. The selection of the area is intentional for only areas which experience high annual rainfall of 1500 mm and above and prone to high risk of flooding due to the heavy rainfall and proximity to the Atlantic Ocean. The area is also close to rivers Niger and Benue and selected based on three agro-climatic zones: mangrove swamps, tropical rainforest, and Guinea Savanna only. This means that a combination of purposeful and stratified sampling was employed in selecting the study area.

Data Sources and Collection Methods

Secondary data was used for this study and includes forecast and observed precipitation data for 21 years from 1995 to 2015 over Nigeria. This is the period for which reforecast data was available. The forecast data is the ECMWF ensemble reforecast precipitation data in mm/day downloaded from the S2S data website hosted by ECMWF and it is available up to lead time of 40 days from 11 ensemble members. The mean ensemble precipitation forecast data provided at a resolution of 1.5°× 1.5° grids (IRI 2016) was used for this study. The observed rainfall data is the CHIRPS reanalysis of daily precipitation provided at a resolution of 0.05 × 0.05 grid which is higher than the resolution of the forecast data. It was downloaded from the IRI website earlier discussed and credited to Funk et al. (2014).

No missing data was observed in the data sets as it is likely that any existing missing data may have been filled during reanalysis/homogenization before making the data available for users on the websites.

Weekly precipitation totals and climatological averages of observation and forecast from May 14 to September 24 (20 weeks) for the 21 year period (1995–2015) was extracted and used to study the intra-seasonal changes in precipitation. The choice of the seasonal period (May–September) is based on the enormous rainfall usually experienced during this period. ENSO years within the period were selected based on the Oceanic Nino index (ONI) provided by NOAA at http://www.cpc.ncep.noaa.gov/products/analysismonitoring/ensostuff/ensoyears.shtml

Data Analysis

Codes were written first, to read all the data files which were in NetCDF format, and then to convert the time variable which was in Julian days in CHIRPS data to the normal calendar dates to enable easy extraction of weekly totals of precipitation from May 14 to September 24. This was then followed by extraction of weekly means and climatological averages of precipitation over southern Nigeria from which temporal plots were made. Time series were plotted as average weekly while the climatological average was determined from the weekly totals averaged over 21 years. Bias and RMSE between forecast and observed precipitation was calculated accordingly. Anomalies were determined by subtracting the climatological mean from the weekly mean precipitation and the anomalies correlated to determine the ACC skill of the forecast at different lead times. All plots as well as calculations were made using Python.

The ECMWF forecast data, unlike CHIRPS, had a land-sea masked file for measurements over land and ocean. This masked file was used to mask out the ocean precipitation values in order to extract only values over land. The step is taken to isolate the land grid boxes when calculating precipitation averages and totals to enable a fair comparison between the data sets.

The observed precipitation for the previous week was persisted for the following week to produce the one week lead time persistent forecast for the 20 weeks period (May 14–September 24). This was then compared with the observed climatology and forecasts at weekly lead times of one, two, three, and four weeks referred to as lead 1, 2, 3, and 4. The El Nino and La Nina events were identified by averaging the five consecutive Oceanic Nino Index (ONI) values of months AMJ, MJJ, JJA, JAS, ASO, and the average was compared with the threshold that defined an ENSO event, that is, ONI values greater or equal to +0.5 °C define an El Nino while values less or equal to −0.5 is La Nina (NOAA 2016). These months were chosen to be consistent with the season (May–September) for which forecast is made. The flow chart (Fig. 2) summarizes the procedure involved.

Fig. 2
figure 2

Flow chart of methodology. (Source: Ugbah 2016)

Full size image

Results and Discussion

Comparison Between Observation and Forecast at Different Lead Times

Weekly totals of the forecast and observed precipitation covering the twenty weeks period (May 14 to September 24) from 1995 to 2015 were plotted to compare the extent to which the observation agrees with or deviates from the forecast at different lead times. The result is shown in Fig. 3.

It can be seen from Fig. 3 that both the forecast and observation follow a similar pattern during the season and depict the two peaks (bimodal) of rainfall in early July and in September. The short period of the little dry season known as “August break” is also shown by the precipitation minimum around late July/August. At initial start dates up to June11 and towards the end of the season in September, the precipitation values all seem to be in close agreement, having only small differences, but beyond the week starting June 11, the forecasts start to deviate significantly from the observed with the largest differences around the period of the little dry season after which the lines begin to converge until the end of the season (Fig. 3). It can also be seen that the differences in precipitation amount increase with increasing lead time as depicted by lead 1 forecast (red line) which lies closest to the observation (dotted black line) throughout the season. Forecast of lead 2, 3, and 4 show larger departures from the observed climatology (Fig. 3) with the largest departure seen in the lead 4 forecast (purple dotted line). These large departures possibly arise from the inability of the model to properly simulate the little dry season phenomenon and related subseasonal processes. Correction of these differences is likely to improve the performance of the model. The plot also shows that the predicted precipitation at all the different lead times is lower than the observed implying an underestimation by the model. This can also be corrected to improve the model.

Fig. 3
figure 3

Forecast at different weekly lead times (wk1-wk4) indicated with colored lines and Observed weekly climatology (black dotted line) compared (Source: Ugbah 2016)

Full size image

Evaluation of the Forecast Skill: Metrics

It is important to explain some of the different scores that are used for evaluating this forecast as there are different types of metrics used for verification of forecast and the choice of a particular type depends on the objective of the verification. Some of the popular metrics used by scientists and recommended by WMO for use in verification of precipitation forecasts include bias, root mean square error (RMSE), and anomaly correlation coefficient (ACC). These metrics have been selected for this study to provide a common standard for comparing the results with those of other scientist and meteorologist on related works.

Bias

The bias refers to the differences between the forecast and the observation mean. It is a measure of the extent to which the ECMWF model under-estimates or over-estimates precipitation. Bias information is useful in correcting errors in models which are likely to improve its performance (Inness and Dorling 2010). Mathematically it is defined as:

$$ \mathrm{Bias}=\sum \limits_{i=1}^n\frac{\left(f-O\right)}{N} $$

where f = forecast, O = observation, and N = total number of events.

A larger bias shows large differences or disagreement between forecast and observation while smaller values suggest closeness or similarities. Such closeness may not necessarily mean that the forecast is perfect because observations could be wrong and therefore give misleading results. Bias can be positive when forecast is over-estimated or negative when under-estimated.

The biases were computed at the four different lead times and then averaged over the twenty weeks period from May to September (Table 1). Negative biases can be seen for leads of 1, 2, 3, and 4 weeks. The biases increase with lead time from week one (−0.69) to week four (−3.10). This is consistent with findings from Vitart 2004, 2014, and Li and Robertson (2015). At lead times of 2 to 4 weeks, the dry bias is particularly strong from late June through July. This suggests difficulty by the model to forecast at a period when reduced rainfall is experienced in the West African Monsoon (WAM) and when the Inter-Tropical Convergence Zone (ITCZ) also called ITD is at its most northward position from the area of study. Such biases may be necessary to correct only if other metrics also provide evidence in support of poor model performance. This is because bias alone is inadequate in evaluating the performance of a model as it only tells only the differences and does not give weight to errors like the RMSE. That is why the use of only one metric is not recommended but a combination of two or more.

Table 1 1995–2015 ECMWF forecast vs observed precipitation scores at weekly lead times
Full size table

Root Mean Square Error (RMSE)

RMSE can be defined as the square root of the sum of the squares of all errors (biases) between a given set of data as shown below (Chai and Draxler 2014).

This metric is good for use in evaluating the performance of a model because it represents normally distributed errors better than uniformly distributed errors common in models (Chai and Draxler 2014). It gives more weight to large errors than smaller errors and has a tendency of showing some bias towards datasets without large errors, but it still remains one of the oldest and most popularly used metric among scientists in model evaluation. Large RMSE values imply large error distribution while smaller values suggest smaller errors. RMSE of zero means that the forecast is without errors or bias.

$$ \mathrm{RMSE}=\sum \limits_{i=1}^N\frac{\left(f-O\right)2}{N} $$

where f = forecast, O = observation, N = total number of events.

The results of RMSE also show that errors increased with lead time, and were lowest (1.64) at lead time of one week and highest (3.98) at lead time of four weeks (Table 1) .

Anomaly Correlation Coefficient (ACC)

ACC was used to evaluate the potential skill of the forecast. It is considered a potential skill because it is only one metric out of the several others that need to be considered to arrive at the final or perfect skill of a forecast. It may not be exhaustive, but it provides very useful information for assessing model performance and a pointer to the likely overall skill. Correlation between forecast and CHIRPS observed daily precipitation data was done using python program scripts written to incorporate the mathematical formula for ACC defined by Maue and Langland (2014) as:

$$ \mathrm{ACC}=\frac{\sum_{m=1}^m{f}^{\prime}\ast {O}^{\prime }}{\sqrt{\left({\sum}_{m=1}^m\left({f}^{\prime}\right)2\ast \sum \limits_{m=1}^m\left({O}^{\prime}\right)2\right)}} $$

where f’ = fm-Cm, is the forecast mean (fm) – climatological mean of forecast (Cm) O′ = Om-Cm, is the mean of observed (Om) – Observed mean Climatology (Cm).

The climatological average values of forecast and observation for the 21 year period was first calculated and then subtracted from each weekly forecast and observed precipitation to get the desired result. Correlation values generated from linear regression analysis of these anomalies also produced similar results when compared. This served as a check for possible computational errors from code written to calculate this metric. One advantage with this metric is the consideration of errors averaged over a longer period than for short period averages which are usually highly variable. This advantage, on the other hand, also constrains the metric in its ability to significantly reflect changes in the mean values over shorter time ranges.

The results of the correlation for weekly start dates during the season and at lead times of the 1, 2, 3, and 4 weeks indicated are further discussed in Sect. Performance of the S2S Model precipitation Forecast During Rainfall Peaks and Period of Little Dry Season. There are 20 weekly start dates in the season starting with the first on May 14, then May 21, May 28, … up to the last date on 24th September.

A forecast can be produced on any of these dates for lead times up to 40 days in advance, but in this study, only lead times up to 4 weeks are considered. This is because such a time period is relatively reliable and adequate for use by farmers especially in Nigeria to plan and make decisions about farming practices such as planting, weeding, fertilizer application, etc. It should be noted also that at longer lead times, forecasts tend to lose accuracy (Inness and Dorling 2010), and so, the weekly to one month lead time forecast chosen is likely to be more reliable than lead time more than one month. In this study, forecast made with a lead time of one week (referred to as lead1) means the forecast is produced on a particular week and is valid for one week (up to the following week). It is also possible to produce a lead 1, 2, 3, or 4 forecast on any of the 20 start dates from May 14 to September 24. The choice of any of the start dates depends on the period of interest within the season and the goal which the user or client wants to achieve or satisfy.

The correlation between the observed and forecast anomalies (ACC) at different lead times and with persistence forecast is shown in Fig. 4.

Fig. 4
figure 4

Anomaly Correlation between forecast at different lead times compared with persistence forecast. Topmost thick horizontal grey dotted line indicates ACC values that are statistically significant at 0.01 while the solid grey line represents values that are significant at 0.05 level (Source: Ugbah 2016)

Full size image

Seasonal mean ACC values tend to be generally poor and less than 0.5 but weekly values tend to be better in some weeks at different lead times. For example, start dates of week 8 (July 2), week 12 (July 30), week 13 (August 6), week 14 (August 13), and week15 (August 20) for lead 1 forecast have the highest and best ACC values of 0.71, 0.84, 0.75, 0.72, and 0.65, respectively, which suggest a strong positive relationship between forecast and observation. These values are all significant at 0.01 level while the ACC value of 0.54 in week 11 (July 23) is significant at 0.05 level. High ACC scores were also observed for a few start dates at lead times 2, 3, and 4. The scores generally decreased with increasing lead time (Fig. 4) and the poorest scores were observed in lead 4 forecasts with a few negative correlations. This is consistent with Li and Robertson (2015). A statistical test to determine the level of significance of these values was done using the Statistical Package for Social Sciences (SPSS) software.

Persistence is also a baseline, like mean climatology, used to compare with forecasts in order to determine their quality. The persistence forecast for lead 1 was produced by persisting observation of the present week for the following week’s forecast. The result was then correlated with the forecast at different lead times. It is often said that the long time range involved in subseasonal forecasting has made it difficult for it to beat Persistence forecast (Vitart 2014) but result from this study has proven otherwise as the forecast at the different lead times beat persistence in most of the weeks This approach according to Jolliffe and Stephenson (2003) only performs well for short range forecast, but result from this work support the view that it can also perform well for medium ranged forecast as shown (Fig. 4).

Comparison Between Observed and Forecast during El Nino and La Nina Years

ENSO and MJO both interact in modulating subseasonal to seasonal precipitation. It is assumed that since the ECMWF model is known for its high skill in simulating MJO events, it is also likely to accurately simulate subseasonal rainfall variability during different phases of ENSO. Based on this, it will be good to look at how the model forecast for precipitation during El Nino and La Nina years either agree or differ from the lead 1 forecast for the 1995–2015 period. Only lead one forecast is chosen because it shows the least bias with observed climatology compared to lead times of two, three, and four weeks. El Nino and La Nina years were identified using the Oceanic Nino Index (ONI) as earlier discussed. Four La-Nina (1998, 1999, 2000, 2010) and Five El-Nino events (1997, 2002, 2004, 2009, 2015) were identified and the forecast for these years extracted and plotted for comparison. The selection of 1997 El Nino and 1999 La Nina events is in agreement with Enso events between 1963–1999 identified by Joly and Voildoire (2009). The strength of ENSO usually weakens towards the end of first quarter of the year and reaches maximum strength around December to February (Joly and Voildoire 2009). A strong African monsoon is linked to La Nina events which are observed in April, May, or June (Joly and Voildoire 2009).

The result of analysis shows that both La Nina and El Nino years forecast are in reasonable agreement with observed precipitation during the first eight weeks (May14–July2) and show a wetter bias, but begin to show marked differences beyond this period particularly in mid-July and in August (Fig. 5) when the little dry season is usually experienced in the study area.

Fig. 5
figure 5

Precipitation forecast and observation for El Nino and La Nina years for one week lead time compared with observed climatology (Source: Ugbah 2016)

Full size image

The differences between them were calculated as bias and compared with the 1995–2015 climatology shown in Table 2.

Table 2 Comparing Lead1 scores for the climatological period, El Nino, and La Nina years
Full size table

The seasonal mean biases were all negative suggesting underestimation of precipitation by the model compared to observation and was higher (−0.97) during La Nina years than El Nino (−0.41). The 1995–2015 period as a whole, however, showed lower bias compared to the La Nina years. It is likely that the larger bias might be by chance rather than a certainty. The El Nino years showed better results with the least bias and RMSE. The seasonal mean RMSE in La Nina years was 2.24 compared to El Nino which has 1.84 (Table 2).

Weekly biases compared were positive during the first 7–8 weeks after which they became negative. This is to say that forecast for both El Nino and La Nina years was over predicted during the initial weeks and under-predicted thereafter. Another observation is that the forecast seem to follow the pattern of climatology rather than observation and was lower than the observed for a greater part of the season. The high bias and RMSE particularly in La Nina years shows that the model might be worst in La Nina years and more reliable in El Nino years as the El Nino forecast tends to agree more with the observation and climatology than forecast during La Nina years. All these still point to the model’s inability to accurately simulate southern Nigeria precipitation and suggest that there is still a missing tele-connection between precipitation in the study area and ENSO that has not yet been properly resolved by the ECMWF model. This, however, does not imply that the model is bad, but requires fine-tuning to correct some of the biases observed in order to improve the quality of the forecast. Such biases could arise from the poor representation of the ENSO, MJO, and precipitation tele-connection within the study area. Another reason could be incorrect parameterization of convection and processes which modulate weather over Nigeria. This can also adjusted to give better results. The timing of ENSO events and the delay in the response of a particular region to an ENSO phase is also an important consideration in simulating the tele-connection between them (Joly and Voildoire 2009). This is because significant tele-connection is sometimes observed during the developing stage of ENSO or during its terminal stage (Joly and Voildoire 2009). All these, still remain a big challenge to many numerical modeling experts trying to simulate tele-connection between ENSO and the West African Monsoon (Joly and Voildoire 2009). The “August break” is well captured by the model forecast with strong agreement during the El Nino years than La Nina. However, the greatest noise is observed during this period (July–August). This again shows that the model can be improved.

Performance of the S2S Model Precipitation Forecast During Rainfall Peaks and Period of Little Dry Season

Should the ECMWF S2S model be able to properly simulate high rainfall amounts and pattern during periods of maximum rainfall within the season, then it will no doubt be a good tool in providing useful information for extreme rainfall disaster preparedness, prevention, and risk reduction. Fortunately, the model is able to properly simulate pattern of rainfall in the region and also identify periods of peak rainfall which mostly occurs in June and September as shown in Figs. 3, 4, and 5. The model is also able to depict the little dry season period which occurs between July and August. Such information is considered useful to planners. But unfortunately the model shows a low skill in simulating rainfall amount especially during the peak rainfall period and the little dry season. This is evident in the large biases observed between last week in June and mid-August at lead time of two, three, and four weeks. The biases and RMSE are however minimal at lead time of one week which is good for early warning purposes and disaster preparedness at least one week before the event happens. The skill is also found to be poor during La Nina years which are notable for high rainfall than El-Nino years in the region. This means that the model is not very skillful in simulating heavy precipitation amounts during heavy rainfall periods especially at lead times beyond one week. It is however still possible that some adjustment in the model by ECMWF to correct the observed biases could give better and reliable results which would make that ECMWF S2S forecast model a dependable tool for predicting extreme high/low rainfall and thus helpful in adapting and mitigating the effects of climate change in the study area.

Conclusion

Large negative biases and root mean square error which were observed to increase with lead times show that the model generally has a dry bias towards estimating precipitation over the area. The high biases and errors observed during La-Nina, than El-Nino years suggests that the model performs better during El-Nino years. Users should therefore be cautious of its use during La-Nina years. Re-calibrating the model output to reduce these biases is likely to produce better results. Seasonal mean Bias and RMSE which generally increased with increasing lead time during the season showed the best results in the one week lead time and the worst in four weeks lead time. This suggests that the forecast was better at shorter lead time. Seasonal mean ACC values which were generally less than 0.5 but with weekly values greater than 0.7 on certain start dates of the lead 1 and 2 forecast, and lower values for leads 3 and 4 confirm that the forecast is best at lead time of one and two weeks during specific periods in the season than at longer lead times. ECMWF should also consider making some adjustments in the model in aspects of parametization of ENSO tele-connection with rainfall over the study area to reduce the large biases observed during La Nina years. This is important because La Nina events bring more rains to the region than El Nino events and so the opportunity of more rains and water availability during La Nina years should be maximally exploited through use of good and reliable prediction tools that will provide accurate and reliable information needed for planning by farmers, environmentalist, disaster managers, water resource managers, and policy makers. The results also show that the S2S model precipitation forecast has a skill which is higher than persistence forecast at all the lead times of one to four weeks, thus negating the statement by Vitart (2014) that the long time range involved in subs-seasonal forecasting has made it difficult for it to beat Persistence forecast.

The observed large negative biases and RMSE at lead times of 2, 3, and 4 weeks suggest that the model is not yet very skillful in simulating precipitation amounts over southern Nigeria at longer time ranges; it is, however, very skillful in simulation precipitation amount, pattern, and periods of rainfall peaks/minima during the season at lead time of 1 week. This is evident in the very low biases and errors, thus confirming that the best results were observed for precipitation forecast made with a lead time of one week only. This means that S2S precipitation forecast can provide one week advance signals of periods of peak rainfall, little dry season and expected rainfall amount over southern Nigeriais. Such information is considered to be a good early warning and disaster preparedness tool. It could also be a useful tool for planning and decision making if provided at least one week before the event happens. Thus, a way of mitigating and adapting to the effects of climate variability and change is used. The ECMWF S2S can therefore be recommended for this purpose and for updating seasonal forecast on weekly basis in Nigeria for maximum Agricultural production and for disaster reduction since the skill is high at lead time of one week. It is therefore a good tool for monitoring of rainfall during the rainy season and could still yield better results at longer lead time if the model is further improved by ECMWF.

Future work will expand the scope to cover the entire Nigeria and also assess the skill of the individual ensemble members of the model instead of the average of the ensembles used in this study. Rain guage observation data will be considered for comparison with forecast. It is also good for work in the future to include determination of rainfall onset, cessation, and length of season using the S2S data instead of looking only at how the model can simulate rainfall amount and pattern as done in this study.