Simulating Survey-Based Consensus Forecasts With Econometrics

Surveying economists for their projections on a variety of economic and financial indicators and aggregating the results has wide appeal, and for good reason. The wisdom of the crowd, such as it is, tends to be more reliable through time compared with any one forecaster. But there are challenges with standard consensus forecasts—challenges that can minimized if not solved with econometric-based applications.

The challenges include the time and expense in surveying and aggregating the data. The Wall Street Journal, for instance, surveys 60-plus economists each month for their expectations on ten key economic indicators. The question is whether there’s an easier, quicker and low-cost methodology to approximate the task with econometric modeling? As we’ll see, there’s a case for answering with a cautious “yes,” or at least a confident “maybe.”

As an example, consider The Capital Spectator’s current estimate for US retail spending in May, which is due to be released later this morning by the Census Bureau. Econoday.com’s consensus forecast sees sales rising by a strong 0.7% vs. the previous month. The survey estimates range from 0.4%-to-0.9%.

For comparison, let’s run the numbers with The Capital Spectator’s combination-forecasting
model,
which uses projections based on aggregated results from nine econometric models. The focus here is on generating estimates for the one-year trend, as summarized in the following chart.

By comparison, one-month projections tend to suffer a higher degree of noise. Although all forecasts tend to miss the mark, the one-year trend is somewhat more reliable as a prediction target as well as more relevant for business-cycle analysis. But if there’s a need to generate a one-month estimate that approximates a conventional consensus forecast, it’s a straightforward task to back-out this data out from a one-year projection.

On that basis, the median point forecast for May’s one-month change in retail spending is a 0.3% gain via The Capital Spectator’s combination-forecasting model. That’s less than half the rate of Econoday.com’s 0.7% point estimate.

Analysis reveals, however, that the backed-out monthly estimates via the combination-forecasting model are often close to the standard point data published by Econoday.com and other services. Surprising? Not really.

Consider that The Capital Spectator’s combination-forecasting model is designed to generate thousands of predictions with nine different models. Those numbers are the raw material for focusing on the median results, for points estimates and prediction intervals. As a long line of research shows, aggregating an array of forecasts tends to outperform individual forecasts. That’s not surprising since the future is always uncertain and no one model or forecaster has a monopoly on robust methods for predicting. The challenge, then, is to minimize the errors, which is the forte of combination forecasts.

Is that a reason to abandon conventional consensus forecasts? No, not necessarily, but at the very least there’s a strong case for supplementing this data with econometric-based estimates.

Keep in mind that econometric modeling offers several advantages over the usual approach to aggregating survey numbers. First, you can generate the data at your convenience vs. waiting for a research service to send you their results, which may be published with a lag or intermittently.

Another plus with econometric modeling: you can update the forecasts frequently and easily. This is especially valuable when something changes that could alter the outlook. Last week’s consensus forecast by your favorite research service may be robust, but a news event can change the economic, financial or political landscape and so it’s prudent to run a fresh set of numbers. That’s not always possible with surveys, but it’s forever an option in real time with econometric models.

Another advantage: econometric forecasts can be applied to data sets that aren’t available with surveys. That reality opens up new opportunities for developing insight that’s otherwise absent with standard-issue consensus forecasts, which focus on a small subset of the world’s economic and financial indicators.

Although econometric modeling can enhance our analytical toolkit for developing informed assumptions about the future, the standard caveat still applies. As the statistician George Box famously observed: “all models are wrong, but some are useful.”

On that basis, approximating consensus with econometric modeling is arguably a better way to mine perspective vs. the standard surveying techniques.