TL;DR
This paper proposes a novel approach to improve inflation forecasts by using machine learning (ML) models to predict and correct human forecasters' errors. The method, akin to an AI-powered grammar checker for forecasts, effectively reduces bias and enhances accuracy, particularly for less skilled forecasters or during periods of high uncertainty.
What Problem Does This Paper Address?
Accurate inflation forecasts are crucial for economic stability and decision-making. While human forecasters are essential, their predictions can be prone to errors due to various factors, including cognitive biases, limited information processing, or periods of structural instability. Existing ML approaches often aim to replace human forecasts entirely or use them as just one input among many. This paper addresses the gap of how ML can augment human forecasts by specifically targeting and correcting their systematic errors, without discarding the valuable insights human forecasters bring.
Methods
The core of the paper's methodology is an "ML-centered forecast adjustment procedure." This involves training ML models (specifically LASSO, elastic net, and ridge regression) to predict the errors of human forecasts. These predicted errors are then used to adjust the original human forecasts. The key novelty lies in this "predict-and-correct" approach, which is designed to be flexible:
- No Restriction on Human Forecasts: The method works regardless of whether human forecasts are purely judgmental or model-based.
- Leverages All Information: The ML models utilize predictors, the actual target variable, and the human forecasts themselves.
- Transparent and Flexible Adjustment: Unlike "black box" models, the ML models are built on standard regression techniques, allowing for interpretation. Human forecasters can choose to partially accept the ML model's suggestions, offering a nuanced adjustment process.
The ML models used are:
- LASSO (Least Absolute Shrinkage and Selection Operator): Known for variable selection and regularization.
- Elastic Net: Combines L1 (LASSO) and L2 (ridge) penalties, offering a balance between selection and shrinkage.
- Ridge Regression: Uses L2 penalty to shrink coefficients, particularly useful when predictors are correlated.
The models are trained to predict forecast errors using macroeconomic and financial variables from the FRED-QD dataset.
Datasets
- US Survey of Professional Forecasters (SPF) Data: This is the primary dataset for human forecasts, specifically quarterly CPI inflation rate forecasts from 1981Q3 to 2024Q3. The study focuses on frequent participants to mitigate missing data issues.
- Federal Reserve Economic Data (FRED-QD): A comprehensive dataset of US macroeconomic and financial variables used as predictors for the ML models.
- Benchmark Forecasts: The authors also generate four sets of benchmark forecasts for comparison:
- Random Walk: A naive benchmark.
- AR(1) Model: A simple autoregressive model.
- Auto ARIMA: A more sophisticated time-series model.
- SPF Consensus: The median forecast from all SPF participants.
Key Insights and Ideas
- AI as a "Grammar Checker" for Forecasts: The analogy to spell-checkers is powerful. It suggests ML can help catch and correct "mistakes" in human forecasts without dictating the entire narrative.
- Human Errors are Predictable: The study demonstrates that systematic errors in human inflation forecasts can be predicted using ML and macroeconomic data.
- Targeting Large Errors is Key: The adjustment procedures are most effective when human forecasts are already inaccurate or biased, suggesting they are particularly beneficial for less skilled forecasters or during volatile periods.
- Bias Reduction is a Significant Benefit: Even when forecast accuracy doesn't improve substantially, the ML adjustments can still reduce forecast bias, which is valuable for economic policy.
- Ridge Regression's Strength in Out-of-Sample: Interestingly, ridge regression, which tends to use more predictors, showed stronger performance in out-of-sample adjustments for large errors, potentially by catching overlooked information.
Key Results
- In-Sample Improvements: Significant improvements in forecast accuracy (MAE and RMSE) and bias reduction were observed in-sample across various benchmarks.
- Out-of-Sample Improvements: While more modest, out-of-sample improvements were still evident, especially for less accurate forecasts. For instance, LASSO adjusted forecasts showed lower MAE for 76% and lower RMSE for 95% of cases in-sample.
- SPF Consensus: The highest quality forecasts (SPF consensus) showed less room for improvement, but even here, some gains were observed.
- Time-Varying Benefits: The adjustments proved particularly helpful during periods of heightened uncertainty, such as the 2008 recession and the post-pandemic inflation surge.
- Large Error Mitigation: The procedures were more effective at reducing large forecast errors (e.g., exceeding 1.5 or 2 standard deviations from the mean). For example, out-of-sample, relative RMSEs for errors exceeding 2 standard deviations were reduced to around 0.87-0.91.
Limitations and Future Work
- Focus on Linear Models: The study primarily uses linear ML models (LASSO, elastic net, ridge). The authors acknowledge that non-linear methods (e.g., random forests) might offer further improvements.
- Data Availability and Hindsight Bias: While FRED-QD mitigates some hindsight bias, the authors acknowledge the inherent challenge of perfectly replicating forecasters' information sets.
- Not a "Beat the SPF" Goal: The aim is not to create a fully automated system that outperforms humans, but rather to assist them.
- Future Work:
- Investigating optimal utilization of predicted errors in forecast combination and ensemble methods.
- Extending the procedures to handle uneven and mixed-frequency data.
- Leveraging the panel structure of the SPF to incorporate information from multiple human forecasters.
- Exploring non-linear ML techniques.
Summary updated