Backcasting the US economy with Dynamic Factor Models
- Using Dynamic Factor Models (DFM) we try to backcast some important US macroeconomic variables
- We find that the composition and interpretation of the factors pre-2008-crisis differ from the post-2008-crisis
- After the crisis, the importance of the factor with housing market variables increases drastically
- Additionally, we evaluate the backcasting accuracy of the DFM on US Industrial Production, Employment and Consumer Prices
- The DFM outperforms the Autoregressive model pre-crisis, but not post-crisis where the DFM only backcasts more accurate in half of the scenarios
- Possible reasons are that there is less data available after the crisis and/or that the structure of the macro economy has changed during the crisis
Co-authored by Elian Griffioen during his internship at RaboResearch.
In this Special, the use of Dynamic Factor Models for analysing and forecasting macroeconomic variables is illustrated with historical monthly US economic data. The DFM is a fully data driven model, that is widely used because it reduces the risk of overfitting present in some other methods. In this study we find that the composition and interpretation of the factors pre-2008-crisis differ from the post-2008-crisis. After the crisis, the housing market factor is the factor that explains the largest share of the variance, while before the crisis it was only the fifth factor. Additionally, we perform a backcasting study on variables Industrial Production, Employment and Consumer Price Index to find the forecasting accuracy of DFM’s in comparison to an autoregressive (AR) model for the observed variables before and after the financial crisis. We find that the DFM outperforms the AR model for all combinations of variables and time horizons before the financial crisis and only for half of the combinations after the financial crisis. Possible explanations can be the lack of available data after the financial crisis and the structure change of the US economy during the crisis.
Macroeconomic forecasting and the case for using Dynamic Factor models
There are multiple ways of forecasting macroeconomic variables like GDP, unemployment and inflation. These vary from simple spreadsheets, via large (structural) models, to fully data driven models. Simple spreadsheet models are often considered too simple to capture all relevant characteristics of the economy and there’s a lot of criticism on large (structural) models, like the famous Lucas critique (Lucas, 1976)
Fully data driven models become increasingly popular. This trend is strengthened by the growing availability of relevant and timely data. A popular example of this kind of model is a Vector Autoregression (VAR) model. However, it is hard to include a large amount of variables in a VAR model without having the risk of overfitting and as a consequence inaccurate forecasts.
Luckily there are data driven models which find solutions for this handicap. Examples of this are Bayesian VAR models (BVAR’s) and Dynamic Factor Models (DFM’s). The former puts restrictions on relationships between variables to prevent overfitting. The latter tries to find a small set of factors that contain the co-movements of a group of explanatory variables and uses these factors to forecast other variables.
In this Special we explore the relevance of DFM’s for macroeconomic forecasting and apply it on historical US data to explain growth in industrial production, consumer prices and employment. A comparable study is performed by Stock & Watson (2002a) to illustrate the forecasting performance of DFM’s. We will compare the results pre- and post-2008-crisis. The results suggest that the structure of the US economy might have changed after the crisis.
Dynamic Factor Models in short
Dynamic Factor Models assume that all variables consist of one or more common components reflecting the shared underlying trends within the set of variables and a variable-specific (idiosyncratic) component. The method tries to find these components or factors from a set of observed variables. In this way, it is possible to summarize the information from a large set of observed variables with a small set of factors, comparable to a business cycle indicator that shows the state of an economy.
The optimal number of factors is based on an information criterion like that of Bai and Ng (2002). The relation between factors and observed variables is often assumed to be linear, however the thesis of Griffioen (2019) explores nonlinear relations between factors and observed variables.
The next step is to estimate a forecasting equation by regressing the variable of interest on a number of its own lags, a number of factors and an intercept. Finally, this estimated model is ready to forecast the variable of interest in future periods.
In addition, the derived factors can usually be interpreted themselves. One of the underlying assumptions is that the economy has a factor structure, which means that a few common trends/factors give an accurate description of the macro economy. An economic interpretation of these factors can be given by investigating the observed variables that are most strongly correlated with the factors. For example, when a factor is strongly correlated to variables that describe different price indices of a country, one can suggest that this factor contains the trend in the prices of a country. In this way, important forces in the economy can be derived. Below we will show that these forces are not constant over time, because there seems to have been a structural break during the 2008-crisis.
We use a DFM approach to explain the variation of the observed variables describing the US economy. We then use the factors to backcast Industrial Production, Employment and Consumer Price Index. These three variables reflect important parts elements in the economy: real economic developments, labour market and the price developments.
We split the history in a pre- and post-crisis period and leave the financial crisis itself (2007-2010) out of scope. The reason for this is that an economy might behave differently during a severe crisis than during ‘normal’ economic circumstances. For example, Alessi et al (2014) show differences between the forecasting performance before and during the financial crisis of macroeconomic variables.
It is even possible that the structure and working of the economy changes because of a crisis. This implies that models estimated on pre-crisis data are not able to forecast the post-crisis economy accurately. We will investigate this further by comparing the factors of the pre- and post-crisis periods.
Variables of interest and available data
The monthly US economic variables are obtained from the dataset of McCracken & Ng (2016) . After treatment of missing values and transformations to stationary versions of the variables, the dataset consists of 714 observations for 121 variables ranging from 1/3/1960 to 1/8/2019. The dataset is split to obtain data before and after the financial crisis. The pre-crisis dataset consists of observations ranging from 1/3/1960 to 1/12/2006; the post-crisis dataset of observations ranging from 1/1/2011 to 1/8/2019. The factors are estimated by Principal Components Analysis (PCA), as explained clearly by Stock & Watson (2002b).
Creating and interpreting the factors pre- and post-crisis
The factor structure of the US economy before and after the crisis can be compared by interpretation of the estimated factors. The factors are estimated on the datasets before and after the financial crisis and the information criterion from Bai & Ng (2002) is used to select the number of factors. The factors can be interpreted from a macroeconomic perspective by inspecting the variance of the observed variables explained by each of the factors. The explained variance of a factor for an observed variable is acquired as the R-squared of a regression of the observed variable on the factor. In the article of McCracken & Ng (2016), the variables are divided into eight groups: Output and Income; Labor Market; Housing; Consumption, Orders and Inventories; Money and Credit; Interest and Exchange Rates; Prices; and Stock Market. Each of the factors is interpreted by inspecting the group labels of the ten variables with the highest explained variance. The interpretation of the factors before the financial crisis are shown in table 1; the interpretation after the financial crisis in table 2.
Before the financial crisis, five factors were found that explain together 38.3 percent of the variance of the observed variables. The first factor can be interpreted as a combination of Output and Income and Labor Market variables and explains 15.3 percent of the variance in the observed variables. This factor can be viewed as a trend in Output and Income and Labor Market variables. The second and fourth factor describe trends in interest and exchange rates. The third and fifth factor consist of a trend in price variables and housing variables respectively.
The results indicate that the factors for the US economy changed during the financial crisis. Six factors are found in the macroeconomic variables after the financial crisis that explain 44.3 percent of the variance of the macroeconomic variables after the financial crisis. Although the type of trends are similar for the factors before and after the crisis, the importance of the factors in explained variance terms is different. The factor with the highest explained variance (11.3 percent) is related strongly to housing variables, while the housing factor is the fifth factor in the data before the crisis with an explained variance of 4.8 percent. This might be due to the large role of the housing market in the financial crisis, which puts it in the centre of the macroeconomy, even post-crisis.
In addition, the third factor from the post-crisis data consists of trends of the output and income variables. In the pre-crisis data, this trend is captured by the most important factor together with the trend in the labor market variables. The importance of the housing trend seems to be increased after the financial crisis while the output and income trend seems to be decreasing.
Modelling and backcasting US macroeconomic variables using a Dynamic Factor Model pre- and post-crisis
Since we saw that the composition of the factors has changed, we expect differences in the forecast performance. To investigate this, we perform an out-of-sample backcasting experiment to compare the forecasting accuracy of the DFM to the autoregressive (AR) model, which is often seen as a simple baseline model, before and after the financial crisis. Backcasting prescribes to forecast historical data with the available data at that moment in time. For example, a 12 month ahead forecast for 1/1/2001 is constructed with a model estimated on observations until 1/1/2000. As the true observed values of these backcasts are known, the backcasting accuracy can be measured and compared for different models.
We measure the backcasting performance of the DFM by comparing the mean square error (MSE) of the DFM and the AR model. We specifically compare the one, six, twelve and twenty-four months ahead backcasts of variables Industrial Production, Employment and Consumer Price Index before and after the financial crisis. Industrial Production and Employment are transformed as I(1) in logarithm and Consumer Price Index as I(2) in logarithms to obtain stationary versions of these variables. The exact transformations can be found in the forecasting study of Stock & Watson (2002a). In the DFM, the backcasting variable is regressed on the factors, the lags of the dependent variable and an intercept and the backcast is produced with these estimated coefficients. In the autoregressive model, the backcasting variable is regressed on only the lags of the dependent variable and an intercept and again the backcast is produced with these coefficients.
To construct a backcast, the following steps are taken: (i) estimate the factors with PCA (only for DFM backcasts), (ii) estimate the backcasting equation, (iii) select the number of lags of the dependent variable with the Bayesian Information Criterion and (iv) construct the backcast. After each backcast, the estimation sample is expanded with one observation and the procedure is repeated. In this way, a set of backcast is obtained for each model which can be compared against the true values of the observations. 420 backcasts are obtained for the data before the financial crisis with a first estimation sample of 118 observations and five estimated factors; 40 backcasts are obtained for the data after the financial crisis with a first estimation sample of 40 observations and six estimated factors.
The DFM provides better backcasts in all scenarios before the financial crisis with differences ranging from 6 to 34 percent (table 1). However, the backcasts after the financial crisis show a much more nuanced view on this comparison (table 2). For each of the backcasting variables, only half of the time horizons shows better backcasts from the DFM compared to the autoregressive model. Industrial Production forecasts from a DFM are more accurate for one and sixth months ahead backcasts, Employment backcasts from a DFM are more accurate for six and twelve months ahead backcasts and Consumer Price Index forecasts from a DFM are more accurate for one and twelve months ahead forecasts.
Additional analyses showed that the backcasting results after the financial crisis are sensitive to the number of observations that are selected. A possible explanation might be the (compared with the pre-crisis period) relatively small sample size used to produce the backcasts. Figure 1 gives a visual representation of the estimated and observed values both pre- and post-crisis for the dependent variables.
Lessons for macroeconomic forecasting
We found that the factor structure of the US economy is different between the pre- and post-financial crisis periods. In more recent data the trend of the housing variables is the most important factor in terms of explained variance, while this factor plays a minor role before the financial crisis. In addition, the backcasting performance of Dynamic Factor Models differs between both periods. The DFM outperforms the autoregressive model before the financial crisis in forecasting Industrial Production, Employment and Consumer Price Index at all forecasting horizons.
After the financial crisis, the DFM provides only better backcasts for half of the time horizons at each variable. This decreased performance might be due to the (compared to the pre-crisis period) relatively small number of available observations to construct the backcasts after the financial crisis. Second, a change in the structure of the US economy after the financial crisis might also be an explanation.
The decreased forecasting performance after the financial crisis does not per se apply to other variables and other countries. For the US economy, there might be other macroeconomic variables which DFM’s can still forecast accurately (e.g. housing market and financial market variables), but this should be tested first. The country differences can be tested further by applying a DFM to, for example, emerging markets or countries that were largely immune for the great financial crisis. In addition, when more post-crisis data become available and a ‘new normal’ economic structure is in place, the forecast performance of DFM’s might again improve to pre-crisis levels for the US.
 In case of overfitting the model contains too many parameters in relation to the number of observations. As a consequence some of the residual variation (‘noise’) is also modelled which worsens the forecasting performance of the model.
 We decided to split the sample in two parts -a pre-crisis sample and a post-crisis sample- to analyse the differences in the factor loadings and model performance. The reason is that by using Monte Carlo simulations it is shown that the number of factors in the model will be systematically overestimated, if one does not take the dynamics of the factor structure change over time into account (Breitung and Eickmeier, 2011). There are also formal ways to test for a break in the factor loadings in the DFM. Stock and Watson (2016) list several break tests, but note one should be careful when interpreting the results. The tests might be sensitive to heteroscedasticity of the residuals and for breaks in the VAR process that the factors are following. Additionally, the parameter instability might not be caused by a break but by different reasons (like drifting parameters).
 This comparison is an often used approach in the literature on economic model evaluation. From this analysis cannot be concluded how the DFM performs compared to other models. It would have been interesting to compare the DFM with the Chicago Fed National Activity Index which uses a similar approach. However, this index acts as an indicator on economic activity and does not give a direct forecast of macroeconomic variables.