Skip to main content

Advertisement

Log in

Nowcasting Real Economic Activity in the Euro Area: Assessing the Impact of Qualitative Surveys

  • Research Paper
  • Published:
Journal of Business Cycle Research Aims and scope Submit manuscript

Abstract

This paper analyses the contribution of survey data, in particular various sentiment indicators, to nowcasts of quarterly euro area GDP. It uses a genuine real-time dataset that is constructed from original press releases in order to transform the actual dataflow into an interpretable flow of news. The latter is defined as the difference between the released values and the prediction of a mixed-frequency dynamic factor model. Our purpose is twofold. First, we aim to quantify the specific value added for nowcasting GDP from a set of heterogeneous data releases including not only sentiment indicators constructed by Eurostat, Markit, the National Bank of Belgium, IFO, ZEW, GfK or Sentix, but also hard data regarding industrial production or retail sales in the aggregate euro area and individually in some of the largest euro area countries. Second, our quantitative analysis is used to draw up an overall ranking of the indicators, on the basis of their average contribution to updates of the nowcast. Among the survey indicators, we find the strongest impact for the Markit Manufacturing PMI and the Business Climate Indicator in the euro area, and the IFO Business Climate and IFO Expectations in Germany. The widely monitored consumer confidence indicators, on the other hand, typically do not lead to significant revisions of the nowcast. In addition, even if euro area industrial production is a relevant predictor, hard data generally contribute less to the nowcasts: they may be more closely correlated with GDP but their relatively late availability implies that they can to a large extent be anticipated by nowcasting on the basis of survey data and, hence, their ‘news’ component is smaller. Finally, we also show that, in line with the previous literature, the NBB’s own business confidence indicator appears to be useful for predicting euro area GDP. The prevalence of survey data remains also under a counterfactual scenario in which hard data are released without any delay. This finding confirms that, in addition to being available in a more timely manner, survey data also contain relevant information that does not seem to be captured by hard data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. As of 29 April 2016, Eurostat also publishes a preliminary flash GDP with a delay of about 30 days. A second, more final, GDP flash estimate will continue to be published about 45 days after the end of the reference quarter. In this paper, we will not take on board the preliminary flash publication, as the time span covered is too limited.

  2. Please refer to Bańbura et al. (2013) for an overview. More recent GDP forecasting examples based on DFM include de Antonio Liedo (2015) for Belgium, D’Agostino and O’Brien (2013) for Ireland, Bragoli et al. (2015) for Brazil, Franta et al. (2016) for the Czech Republic, Modugno et al. (2016) for Turkey, Bragoli and Modugno (2017) for Canada and Bragoli (2017) for Japan. Also, the EUROMIND model developed by Frale et al. (2011) models the individual components of euro GDP separately.

  3. JDemetra+ is free and open-source software written in Java. Download it here: https://github.com/jdemetra/jdemetra-app/releases. The Nowcasting plugin can also be downloaded here: https://github.com/nbbrd/jdemetra-nowcasting/releases. After downloading it, go to the Tools option in J Demetra+ and select plugins and proceed with the installation. The software is portable and platform independent so it could even be launched using any operating system from a USB key.

  4. In case revisions have occurred to past data, the year-on-year growth rate is likely to give some information on this as well.

  5. Interestingly, Camacho and Pérez-Quirós (2010) do acknowledge that there is a problem with the use of vintages. For only one of the variables, GDP, they build a time series composed of flash estimates, and a different time series containing revised values. Such refinement, however, is not considered for the remaining time series. De Antonio Liedo (2015), who places more emphasis on the Belgian economy than on the euro area, follows the same approach, but also fails to reconstruct the monthly releases of hard data.

  6. This update takes the same form of a simple OLS regression of the factors on the innovations. Note that the size of the news vector \( V_{t+1}\) may be large in practical applications when many variables are released at the same time or many observations for the same variable are made available simultaneously. However, the resulting gain remains a function of a reduced number of model parameters \(\theta \).

  7. Bańbura and Modugno (2010) propose a general notation in their Sect. 2.3. and use it to expand expression 4.

  8. The formulation we are using for the case where we have only two news components can be easily generalised to handle more realistic situations where we have a larger set of news concerning different reference dates. Expression 5 will simply grow to incorporate the relevant precision matrix multiplied by the corresponding loadings. If the news refers to different periods of time, the only change relates to the time indices in the second parenthesis. As mentioned earlier, the increased complexity does not prevent us from exploiting the Kalman smoother.

  9. The nowcasting plugin of JDemetra+ contains visual tools such as the schemaball to inspect for correlation patterns. If instead of having a few measurement errors correlated with each other we identify a pattern of cross-correlation that affects most of the measurement errors in the panel, it would be impossible to distinguish whether the correlation in the data comes from the common factors or from the measurement errors. Thus, estimates would not be consistent in this case.

  10. The approximation proposed by Mariano and Murasawa (2003) is applied to the factors. Let \(F_{t}\) be the monthly level of the economy and let \(f_{t}=ln F_{t}-ln F_{t-1}\) be its monthly growth rate. Now, define \(F^{Q}_{t}\) as the geometric mean of the last three levels. This implies that \(ln F^{Q}_{t} = \frac{1}{3}(ln F_{t}+ln F_{t-1}+ln F_{t-2})\). The resulting quarterly growth rate of the factors, which we denote as \(f^{Q}_{t}\), can be expressed as \(ln F^{Q}_{t}-ln F^{Q}_{t-3}\). By substituting both terms by the geometric mean approximation we obtain \(f^{Q}_{t}=\frac{1}{3}(ln F_{t}-ln F_{t-3})+\frac{1}{3}(ln F_{t-1}-ln F_{t-4})+\frac{1}{3}(ln F_{t-2}-ln F_{t-5})\). Finally, a simple expression for the quarterly growth rate of the factors in terms of their monthly growth rates can be obtained as follows: \(f^{Q}_{t}=\frac{1}{3}(f_{t}+f_{t-1}+f_{t-2})+\frac{1}{3}(f_{t-1}+f_{t-2}+f_{t-3})+\frac{1}{3}(f_{t-2}+f_{t-3}+f_{t-4})\). Rearranging terms yields the expression \(f^{Q}_{t}={\frac{1}{3} f_{t}} +\frac{2}{3}{f_{t-1}} + {f_{t-2}} +\frac{2}{3}{f_{t-3}} +\frac{1}{3}{f_{t-4}}\) presented above.

  11. Series that behave erratically or are not significantly correlated with the factors would yield a poor forecast. Specifying an ARMA process for the measurement component of those series would clearly improve the forecast, but this does not change the fact that the correlation of those series (and hence, their news component) with the factors is small. Thus, the weight associated with those series is likely to remain low.

  12. Our actual model uses 4 factors instead of one and some surveys are linked to the cummulative sum of the factors over 12 months, as in Camacho and Pérez-Quirós (2010).

  13. Numerical optimisation of the likelihood, which is feasible for parsimonious models, has the advantage that it does not require the Kalman smoother. Moreover, the multithreading ability of most software packages is able to reduce the execution time by exploiting multiple processors. For example, the current estimation of dynamic factor models in JDemetra\(+\) is feasible without the need to apply the EM algorithm.

  14. The original model is maintained, modified and updated by the Federal Reserve Bank of Philadelphia.

  15. Please refer to the publication by Williamson (2015) of Markit Economic Research entitled ‘Using PMI survey data to predict official eurozone GDP growth rates’ to find out more about these PMI-implied growth rates.

  16. Oddly, the impact of the GfK consumer confidence even displays the wrong sign: a positive surprise in consumer confidence actually leads to a downward revision of the GDP nowcast. We believe that this result should rather be interpreted as a zero impact.

  17. This was less of a risk in the benchmark case, because the annual growth rate refers to a level of industrial production of one year ago, which already incorporates some revision. Hence, while the monthly growth rate only provides information on the most recent observation, the annual growth rate already gives an indication of past revisions.

  18. See Koopman and Harvey (2003) for details on the calculation of observation weights.

  19. Camacho and Pérez-Quirós (2010), for example, exploited the same idea and found, within the context of their ‘Eurosting’ model, that the euro area GDP forecast obtained on 24 January 2007 for the first quarter of that year was fully driven by the NBB Business Survey, simply because it was the only indicator available for January. This can be misleading because that figure does not necessarily change the forecast and it could have been anticipated to some extent on the basis of other indicators that were available.

References

  • Abberger, K. (2007). Qualitative business surveys and the assessment of employment—A case study for Germany. International Journal of Forecasting, 23, 249–258.

    Article  Google Scholar 

  • Angelini, E., Camba-Mendez, G., Giannone, D., Rünstler, G., & Reichlin, L. (2011). Short-term forecasts of euro area GDP growth. The Econometrics Journal, 1, C25–C44.

    Article  Google Scholar 

  • Aruoba, S. B., Diebold, F. X., & Scotti, C. (2009). Real-time measurement of business conditions. Journal of Business and Economic Statistics, American Statistical Association, 27(4), 417–427.

    Article  Google Scholar 

  • Bańbura, M., Giannone, D., Modugno, M., & Reichlin, L. (2013). Nowcasting and the real-time data flow. In G. Elliot & A. Timmermann (Eds.), Handbook of economic forecasting (Vol. 2). Amsterdam: Elsevier.

    Google Scholar 

  • Bańbura, M., Giannone, D., & Reichlin, L. (2011). Nowcasting. In M. P. Clements, & D. F. Hendy (Eds.)Oxford handbook on economic forecasting.

  • Bańbura, M., & Modugno M. (2010). Maximum likelihood estimation of factor models on data sets with arbitrary pattern of missing data. In ECB working paper series, 1189

  • Bańbura, M., & Rünstler, G. (2011). A look into the factor model black box: Publication lags and the role of hard and soft data in forecasting GDP. International Journal of Forecasting, 27(2), 333–346.

    Article  Google Scholar 

  • Bragoli, D. (2017). Now-casting the Japanese economy. International Journal of Forecasting, 33, 390–402.

    Article  Google Scholar 

  • Bragoli, D., Metelli, L., & Modugno, M. (2015). The importance of updating: Evidence from a Brazilian nowcasting model. OECD Journal: Journal of Business Cycle Measurement and Analysis, 1, 5–22.

    Google Scholar 

  • Bragoli, D., & Modugno, M. (2017). A now-casting model for Canada: Do U.S. variables matter? International Journal of Forecasting, 33, 786–800.

    Article  Google Scholar 

  • Camacho, M., & Pérez-Quirós, G. (2010). Introducing the euro-sting. Journal of Applied Econometrics, 25, 663–694.

    Article  Google Scholar 

  • Claveria, O., Pons, E., & Ramos, R. (2007). Business and consumer expectations and macroeconomic forecasts. International Journal of Forecasting, 23, 47–69.

    Article  Google Scholar 

  • Coroneo, L., & Iacone, F. (2016). Comparing predictive accuracy in small samples using fixed-smoothing asymptotics. In Discussion papers University of York, 15/15.

  • Croushore, D., & Stark, T. (2002). Forecasting with a real-time data set for macroeconomists. Journal of Macroeconomics, 24, 507–531.

    Article  Google Scholar 

  • D’Agostino, K. Mc Quinn, & O’Brien, D. (2013). Nowcasting Irish GDP. OECD Journal: Journal of Business Cycle Measurement and Analysis, 7, 21–31.

    Google Scholar 

  • de Antonio Liedo, D. (2015). Nowcasting Belgium. Eurostat review on National Accounts and Macroeconomic Indicators, 75, 7–48.

    Google Scholar 

  • Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 39(1), 1–38.

    Google Scholar 

  • Diebold, F. X., & Mariano, R. S. (1995). Comparing predictive accuracy. Journal of Business and Economic Statistics, 13, 253–265.

    Google Scholar 

  • Diron, M. (2008). Short-term forecasts of euro area real GDP growth: An assessment of real-time performance based on vintage data. Journal of Forecasting, 27, 371–390.

    Article  Google Scholar 

  • Doz, C., Giannone, D., & Reichlin, L. (2012). A quasi maximum likelihood approach for large approximate dynamic factor models. Review of Economics and Statistics, 94, 1014–1024.

    Article  Google Scholar 

  • Durbin, J., & Koopman, S. J. (2001). Time series analysis by state space methods. Oxford: Oxford University Press.

    Google Scholar 

  • European Central Bank. (2011). The measurement and prediction of the euro area business cycle. Monthly Bulletin, May 2011.

  • Evans, M. D. D. (2005). Where are we now? Real-time estimates of the macroeconomy. International Journal of Central Banking, 1, 127–175.

  • Frale, C., Marcellino, M., Mazzi, G. L., & Proietti, T. (2011). EUROMIND: A monthly indicator of the euro area economic conditions. Journal of the Royal Statistical Society: Series A, 174, 439–470.

    Article  Google Scholar 

  • Franta, D., Havrlant, D., & Runsák, M. (2016). Forecasting Czech GDP using mixed-frequency data models. Journal of Business Cycle Research, 12, 165–185.

    Article  Google Scholar 

  • Gayer, A., Girardi, A., & Reuter, A. (2015). The role of survey data in nowcasting euro area GDP growth. Journal of Forecasting, 35, 400–418.

    Google Scholar 

  • Giannone, D., Reichlin, L., & Simonelli, S. (2009). Nowcasting euro area economic activity in real time: The role of confidence indicators. National Institute Economic Review, 210, 90–97.

    Article  Google Scholar 

  • Giannone, D., Reichlin, L., & Small, D. (2008). Nowcasting: The real-time informational content of macroeconomic data releases. Journal of Monetary Economics, 55, 665–676.

    Article  Google Scholar 

  • Harvey, D., Leybourne, S., & Newbold, P. (1997). Testing the equality of prediction mean squared errors. International Journal of Forecasting, 13, 281–291.

    Article  Google Scholar 

  • Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 82, 35–45.

    Article  Google Scholar 

  • Koopman, S. J. & Harvey, A. (2003) Computing observation weights for signal extraction and filtering. Journal of Economic Dynamics and Control, 27, 1317–1333.

  • Kiefer, N. M., & Vogelsang, T. J. (2005). A new asymptotic theory for heteroskedasticity-autocorrelation robust tests. Econometric Theory, 21(6), 1130–1164.

    Article  Google Scholar 

  • Kishor, N. K., & Koenig, E. F. (2012). VAR estimation and forecasting when data are subject to revision. Journal of Business and Economic Statistics, 30, 182–190.

    Google Scholar 

  • Lui, S., Mitchell, J., & Weale, M. (2011). Qualitative business surveys: Signal or noise? Journal of the Royal Statistical Society: Series A, 174, 327–348.

    Article  Google Scholar 

  • Mariano, R. S., & Murasawa, Y. (2003). A new coincident index of business cycles based on monthly and quarterly series. Journal of Applied Econometrics, 18, 427–443.

    Article  Google Scholar 

  • Martinsen, K., Ravazzolo, F., & Wulfsberg, F. (2014). Forecasting macroeconomic variables using disaggregate survey data. International Journal of Forecasting, 30, 65–77.

    Article  Google Scholar 

  • Modugno, M., Soybilgen, B., & Yazgan, E. (2016). Nowcasting Turkish GDP and news decomposition. International Journal of Forecasting, 32, 1369–1384.

    Article  Google Scholar 

  • Newey, W. K., & West, K. D. (1987). A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica, 55(3), 703–708.

    Article  Google Scholar 

  • Patton, A. J., & Timmerman, A. (2012). Forecast rationality tests based on multi-horizon bounds. Journal of Business and Economic Statistics, 30(1), 1–17.

    Article  Google Scholar 

  • Piette, C., & Langenus, G. (2014). Using BREL to now-cast the Belgian business cycle: The role of survey data. Economic Review. National Bank of Belgium, pp. 75–98.

  • Rünstler, G. (2016). On the design of datasets for forecasting with dynamic factor models. In ECB Working Paper Series, 1893.

  • Shumway, R., & Stoffer, D. (1982). An approach to time series smoothing and forecasting using the EM algorithm. Journal of Time Series Analysis, 3, 253–264.

    Article  Google Scholar 

  • The Wall Street Journal. (1999). Euroland discovers a surprise indicator: Belgian confidence, 14 July.

  • Williamson, C. (2015). Using PMI survey data to predict official eurozone GDP growth rates. Markit Economic Research.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to David de Antonio Liedo.

Appendices

Appendix 1: Evaluating Forecasting Accuracy

The prediction errors are defined with a reference i to the information set available at the time the forecast was made:

$$\begin{aligned} e_{t|i}=y_{t}-\hat{y}_{t|\mathcal {F}_{i}} \end{aligned}$$
(11)

where \(\mathcal {F}_{i}\) need not only include lags of \(y_{t}\). In practice, the information that will be actually used may be a small subset of \(\mathcal {F}_{i}\).

The properties of these forecast errors can be assessed in isolation or relative to a benchmark, which we will define as \(\breve{e}_{t|i}\). The benchmark may be a naive forecast, e.g. random walk, in which case \(\breve{y}_{t|\mathcal {F}_{i}}\) would be equal to \(\breve{y}_{t|y_{t-1}}=y_{t-1}\). However, the benchmark could also be a prediction that is regularly published by a forecasting institute or market analysts, i.e. Bloomberg, which is not necessarily model-based. In that case, \(\breve{y}_{t|\mathcal {F}_{i}}\) would be given by methods and a subset of \(\mathcal {F}_{i}\) which is unknown to us.

For model-based forecasts, we use the following notation: \(\hat{y}_{t|\mathcal {F}_{i}}=E_{\theta }[y_{t}|\mathcal {F}_{i}]\) to highlight the fact that they are based on model-consistent expectations given by the parameter vector \(\theta \).

In forecasting comparisons involving competing forecasts resulting from the same information set, the subindex i will be removed because it does not play a role. We will first test the following hypotheses involving forecast errors:

$$\begin{aligned} Unbiasedness:\,&E[e_{t}]=0 \end{aligned}$$
(12)
$$\begin{aligned} Autocorrelation:\,&E[e_{t}e_{t-1}]=0 \end{aligned}$$
(13)
$$\begin{aligned} Equality \,in \,squared\, errors:\,&E[e^2_{t}-\breve{e}^2_{t}]=0 \end{aligned}$$
(14)
$$\begin{aligned} Equality \,in \,absolute\, errors:\,&E[|e_{t}|-|\breve{e}_{t}|]=0 \end{aligned}$$
(15)
$$\begin{aligned} Forecast \,\hat{y}_{t}\, encompasses \,\breve{y}_{t}:&E[(e_{t}-\breve{e}_{t})e_{t}]=0 \end{aligned}$$
(16)
$$\begin{aligned} Forecast \,\breve{y}_{t} \, encompasses \,\hat{y}_{t}:&E[(\breve{e}_{t}-e_{t})\breve{e}_{t}]=0 \end{aligned}$$
(17)

An overview of the tests can also be found in Table 2.

Table 2 Forecasting evaluation tests

1.1 Diebold-Mariano Test

The test originally proposed by Diebold and Mariano (1995) considers a sample path of loss differentials \(\{d_{t}\}^{T}_{t=1}\). In the case of a squared loss function, we have \(d_{t}=e^2_{t}-\breve{e}^2_{t}\). Under the assumption that the loss differential is a covariance stationary series, the sample average, \(\bar{d}\), converges asymptotically to a normal distribution:

$$\begin{aligned} \sqrt{T} \bar{d}\,\,\, \underrightarrow{d}\,\,\, N(\mu , 2\pi f_{d}(0)) \end{aligned}$$
(18)

In particular, they proposed to test the null hypothesis that the forecast errors coming from the two forecasts bring about the same loss: \(E[e^2_{t}-\breve{e}^2_{t}]=0\) against the two-sided alternative. Thus, the resulting p-values represent the probability of obtaining the realized forecast error differential or a more extreme one in a new experiment if the null hypothesis was actually true. The test-statistic that will be used to calculate our p-values is computed as follows:

$$\begin{aligned} DM=\dfrac{\bar{d}}{\sqrt{\dfrac{2\pi \hat{f}_{d}(0)}{T}}} \end{aligned}$$
(19)

where \(2\pi \hat{f}_{d}(0)\) is a consistent estimate of the variance of \(\bar{d}\). Consider \(2\pi \hat{f}_{d}(0)=\sum ^{(T-1)}_{\tau =-(T-1)}w_{\tau }\gamma _{d}(\tau )\), where \(\gamma _{d}(\tau )=\dfrac{1}{T}\sum ^{T}_{t=|\tau |+1}(d_{t}-\bar{d})(d_{t-|\tau |}-\bar{d})\). Under the assumption that \(\gamma _{d}(\tau )=0\) for \(\tau \ge h\), we can use a rectangular lag window estimator by setting \(w_{\tau }=0\) for \(\tau \ge h\). Another option is to use the Heteroscedasticity and Autocorrelation Consistent (HAC) estimator proposed by Newey and West (1987). In this case, the weights could be given by a triangular window, \(w_{\tau }=1-\dfrac{\tau }{h}\) for \(\tau <h \). In this case, however, the consistency property only remains valid when the truncation lag h or bandwidth is a function of the sample size T.

The idea is to test the statistical significance of the regression of \(e^2_{t}-\breve{e}^2_{t}\) on an intercept. In order to determine the statistical significance of the intercept, its associated standard errors need to take into account the autocorrelation patterns of the regression error, which are considered in the denominator of Eq. (19). JDemetra+ exploits the same unified framework to conduct all tests listed in Table 2. But given the small sample sizes that are typical in real-time forecasting applications, which leads to an over-rejection of the null hypothesis, we follow the fixed-smoothing asympotics proposed by Coroneo and Iacone (2016) exploiting the finite sample distributions of Kiefer and Vogelsang (2005). The distribution of the test statistic (19) will depend on kernel (triangular in our case) and the bandwidth chosen, which is set by default equal to \(T^{0.5}\), as suggested by Coroneo and Iacone (2016). The results can be very different than those resulting from the traditional asymptotic theory, where the test statistic would have the same distribution under the null independently of the kernel and the bandwidth used.

Tables 3 and 4 contain the results of this test together with the encompassing test and two efficiency tests, which are described below.

1.2 Encompassing Test

Independently of whether the null hypothesis \(E[e^2_{t}-\breve{e}^2_{t}]=0\) is rejected or not, it is relevant to understand to what extent our model encompasses all the relevant information of the benchmark, and the other way around. Because of the obvious symmetry of both statements, we consider only the first one. If our forecast \(y_{t|\mathcal {F}_{i}}\) encompasses a given benchmark \(\breve{y}_{t|\mathcal {F}_{i}}\), the difference between those benchmark forecasts and ours will not be a relevant factor in explaining our own forecast error. In other words, the regression coefficient \(\lambda \) will not be significantly different from zero in the following regression:

$$\begin{aligned} \underbrace{y_{t}-y_{t|\mathcal {F}_{i}}}_{e_{t}}= \,& {} \lambda \underbrace{(\breve{y}_{t|\mathcal {F}_{i}}-y_{t|\mathcal {F}_{i}})}_{e_{t}-\breve{e}_{t}}+ \xi _{t} \end{aligned}$$
(20)
$$\begin{aligned}&\Updownarrow&\nonumber \\ y_{t}=\,&\lambda \breve{y}_{t|\mathcal {F}_{i}}&+ (1-\lambda ) y_{t|\mathcal {F}_{i}}+ \xi _{t} \end{aligned}$$
(21)

Following Harvey et al. (1997), the statistical significance of the \(\lambda \) coefficient in regression 20 can be used to reject the null hypothesis that our model encompasses the benchmark. In this case of rejection, Eq. (21) suggests that a combination of the two forecasts would yield a more informative forecast.

By construction, the value of the coefficient of a regression \(\breve{e_{t}} =\alpha (\breve{e_{t}}-e_{t})+\xi _{t}\) is equal to \(1-\lambda \), but it is not necessarily true that the rejection of the null hypothesis in the first case implies the acceptance of the symmetric statement.

The test-statistic is computed as follows. When the null hypothesis is that our model encompasses the benchmark, we define the sequence \(\{d_{t}\}^{T}_{t=1}\), where \(d_{t}=e_{t}(e_{t}-\breve{e_{t}})\), and we compute \(E1=\dfrac{\bar{d}}{\sqrt{\dfrac{2\pi \hat{f}_{d}(0)}{T}}} \), exactly as in Eq. (19).

1.3 Efficiency: Bias Test

In order to assess whether our forecasts are unbiased, we will simply test the statistical significance of the average error. In some cases, the time series of forecast errors \(\{e_{t}\}^{T}_{t=1}\) may be autocorrelated to some extent even when they are based on a model with IID innovations. In such cases, the variance associated to the estimate of the average forecast error may be large. The test statistic has exactly the same form as the previous tests discussed so far.

1.4 Efficiency: Autocorrelation Test

We will test here a second necessary condition for our forecasts to be efficient: absence of autocorrelation. In the same spirit as the tests described above, we will assess the statistical significance of the forecast errors’ autocorrelation. Thus, our sequence \(\{d_{t}\}^{T}_{t=1}\) will be defined with \(d_{t}=e_{t}e_{t-1}\).

1.5 Testing the Rationality of Nowcasting Updates

Patton and Timmerman (2012) suggest testing whether the mean squared forecast error is actually decreasing when the horizon decreases. This idea could be applied in our set-up by replacing the concept of forecast horizon with the number of days from the moment in which we update the forecasts for GDP until the day it is realized, i.e. the release date.

In our set-up, the size and power of that test would be too much dependent on the number of times we update the model. We can update it every time we have a new data release, or update it every two weeks, for example. However, what is relevant for us is not whether the model produces rational multi-horizon forecasts, which is likely because they are based on a unique model with parameters obtained via maximum likelihood. Instead, we ask what are the forecasting updates that are most likely to yield significant improvements in forecasting accuracy. The results are available in Table 3.

1.6 Application: Evaluating the Forecasts of Our DFM

The tests described in Table 2 are applied to two cases. In the first case, the aim is to determine which news blocks lead to a significant improvement of the forecasting accuracy. Results displayed in Table 3 include bias, autocorrelation, RMSE and the \(\lambda \) coefficient defined above, which is the weight given to a benchmark forecasts that competes with our model’s. Statistical significance is highlighted with shades. Grey shaded areas in column FS-DM demonstrate which news blocks have induced a significant change in the RMSE of the model, i.e. the null of equal accuracy between old (O) and updated (U) forecasts is rejected. The outcome of the DM test may be considered jointly with the results of the encompassing tests. For a certain news block to be considered relevant, the corresponding nowcasting update (U) should hold a larger amount of information than the older nowcast (O) based on the previous information set, while the old nowcast does not incorporate any useful information absent in the new update. The last two colums of the table show that this is generally the case, with some exceptions. That is, the null U encompases O is not rejected while O encompasses U is rejected.

In the second case, displayed in Table 4, we compare the forecasting accuracy of our dynamic factor model (labeled DFM in the table) with that of relevant benchmarks in the field (e.g. now-casts from the web based service Now-Casting.com, PMI-based forecasts and Bloomberg expectations). Once again, the DM test may be considered together with the encompassing test. Ideally, the nowcasts from our DFM should encompass the information contained in competing forecasts, and not the other way round. Thus, the grey shaded areas in the first column (i.e. DFM encompasses Benchmark) show that the null hypothesis can be always rejected and therefore it is not true that the competitors do not add value. However, the inverse also holds (Benchmark encompasses DFM): the null that the benchmark nowcasts encompass our DFM forecast is also rejected, with only one exception. Hence, the forecasting accuracy of the now-casts could possibly be improved by combining the two information sets together.

Table 3 Statistical significance of each update based on fixed-smoothing (FS) asymptotics
Table 4 Our DFM compared to competitive benchmarks

Appendix 2: Robustness Exercise

1.1 Standard Impacts When the Target Becomes German GDP

In this section, we re-calculate the standard impacts depicted in Fig. 14 and the resulting ranking in Fig. 6 in the case that our target is German flash GDP instead of the euro area flash. The ranking of indicators is shown in Fig. 12. The top four of best-ranked indicators remains unchanged, lead by the Markit PMI. Industrial production in the euro area loses a few spots in the ranking, but remains in the top ten. Industrial production in Germany is now following more closely that of the euro area, in terms of ranking. The NBB Business Confidence has moved to the sixth position after the IFO Business Climate and Expectations for Germany (Fig. 12).

Fig. 12
figure 12

Ranking according to the standard impacts for German GDP. Only the twelve highest-ranked indicators are shown

1.2 Appendix 3: Additional Tables and Figures

See Table 5 and Figs. 13, 14.

Table 5 Dataset
Fig. 13
figure 13

Detailed view of the real-time dataflow. This figure represents the real-time dataflow. All data releases published between June 2015 and October 2015 are represented chronologically. Most of the surveys for each month or reference period are published before the end of the month. This means that the publication dates marked with triangles often fall inside the bar representing reference periods. In the extreme, some variables’ with a strong expectations components are released prior to the reference period. Conversely, variables subject to publication lags, such as GDP or industrial production, will have the triangle way above the reference period. Hard data such as industrial production and retail sales are included in both month-on-month and year-on-year growth rates, except for Spanish retail sales, which are only included in month-on-month growth rates

Fig. 14
figure 14

Standard impacts for euro area GDP Flash. This figure represents the weigths associated to the real-time newsflow multiplied by the standard deviation of the news

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Basselier, R., de Antonio Liedo, D. & Langenus, G. Nowcasting Real Economic Activity in the Euro Area: Assessing the Impact of Qualitative Surveys. J Bus Cycle Res 14, 1–46 (2018). https://doi.org/10.1007/s41549-017-0022-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41549-017-0022-9

Keywords

JEL Classification

Navigation