TL;DR
This paper explores how to best combine predictions from multiple models to forecast foreign exchange rates. While theoretically, Bayesian averaging should be optimal, the authors find it performs poorly in practice due to sensitivity to posterior probability estimates. They propose a novel, heuristic "LogB" method that uses log-posteriors as weights, achieving superior results.
What Problem Does This Paper Address?
Predicting foreign exchange (forex) rates is a notoriously difficult task. The paper tackles the challenge of improving these predictions by combining the outputs of multiple forecasting models. Standard practice often involves simple averaging (e.g., uniform weights), but statistical learning theory suggests that a more sophisticated approach, like Bayesian model averaging (BMA), should yield better results. BMA assigns weights to models based on their posterior probability, reflecting how well they explain the observed data. However, the authors observe that applying BMA directly to forex prediction doesn't deliver the expected improvements, leading to the question: why does the theoretically optimal method fail in practice, and how can we do better?
Methods
The paper compares several model averaging techniques:
- Uniform Averaging: Assigns equal weight to all models. This is a common, practical approach.
- Bayesian Averaging (BMA): Weights models by their posterior probabilities. This is the theoretically optimal approach according to Bayesian principles.
- Bayesian Averaging with Posterior Probability Estimates (BPr): An attempt to address BMA's issues by averaging over the probability estimates themselves.
- Logarithmic Bayesian Averaging (LogB): A novel, heuristic method proposed by the authors. Instead of using raw posterior probabilities as weights, it uses the logarithms of these probabilities. This is a departure from the theoretical foundation but aims to mitigate the sensitivity issues observed with BMA.
The core of the problem lies in how these weights are derived and applied. The authors investigate the distribution of posterior probabilities and how their sensitivity to training data can lead to one model dominating the average, effectively negating the benefits of averaging.
Datasets
The study uses daily foreign exchange data for four currencies from the Chicago Mercantile Exchange. The data spans from 1973 to 1987.
- Training Set: The first ten years of data (1973-1983), comprising 2341 examples.
- Testing Set: The remaining five years of data (1984-1987).
The authors note that a maximum of two weeks was used for the parameter 't' in the technical trading rules, which defines the lookback period for the moving averages.
Key Insights and Ideas
- Bayesian Averaging's Practical Pitfall: The theoretically optimal Bayesian averaging method struggles in practice for forex prediction. This is because the posterior probabilities of the models tend to be highly skewed, with one model dominating, thus preventing true averaging.
- Sensitivity to Posterior Estimates: The formulas for BMA are extremely sensitive to small variations in the probability estimates derived from the training data. This sensitivity can lead to unstable and suboptimal results.
- Heuristic Solution: Logarithmic Weights: A simple, heuristic modification—using the logarithms of posterior probabilities as weights—proves remarkably effective. This "LogB" method bypasses the direct sensitivity issues of BMA while still capturing the relative strengths of different models.
- No Single "Right" Rule: The nature of forex markets, with many "technical rules" showing similar, albeit imperfect, predictive power, makes model averaging a sensible approach.
Key Results
The paper presents results in terms of the five-year return on investment (ROI) obtained from applying different prediction methods.
- Baseline: Simply buying the foreign currency and holding it for five years yielded a 29.5% ROI for the DM currency.
- Single Best Rule: Applying the single best-performing technical rule improved ROI significantly, reaching 47.2% for DM.
- Uniform Averaging: Generally improved upon the single best rule, achieving 52.4% ROI for DM.
- Bayesian Averaging (Bayes): Performed similarly to the single best rule (47.2% for DM), failing to deliver the expected gains due to the skewed posterior distribution issue.
- Bayesian Averaging with Posterior Estimates (BPr): Showed no improvement over standard BMA.
- Logarithmic Bayesian Averaging (LogB): Achieved the best results across all currencies, with DM reaching 52.9% ROI. This method consistently outperformed all other averaging techniques.
Limitations and Future Work
- Heuristic Nature of LogB: The authors acknowledge that the LogB method is heuristic and has lost its theoretical Bayesian foundation. While it performs well, its underlying principles are not as rigorously established as BMA.
- Domain Specificity: The study was conducted specifically on foreign exchange prediction. The authors plan to investigate whether the LogB approach is also useful in other domains.
- Sensitivity of BMA: The core limitation identified is the extreme sensitivity of BMA to probability estimates, which is a significant hurdle for its practical application in noisy domains like financial markets.
Summary updated