CAMul: Calibrated and Accurate Multi-view Time-Series Forecasting

Harshavardhan Kamarthi, Lingkai Kong, Alexander Rodríguez, Chao Zhang, B. Aditya Prakash

Abstract

Probabilistic time-series forecasting enables reliable decision making across many domains. Most forecasting problems have diverse sources of data containing multiple modalities and structures. Leveraging information as well as uncertainty from these data sources for well-calibrated and accurate forecasts is an important challenging problem. Most previous work on multi-modal learning and forecasting simply aggregate intermediate representations from each data view by simple methods of summation or concatenation and do not explicitly model uncertainty for each data-view. We propose a general probabilistic multi-view forecasting framework CAMul, that can learn representations and uncertainty from diverse data sources. It integrates the knowledge and uncertainty from each data view in a dynamic context-specific manner assigning more importance to useful views to model a well-calibrated forecast distribution. We use CAMul for multiple domains with varied sources and modalities and show that CAMul outperforms other state-of-art probabilistic forecasting models by over 25% in accuracy and calibration.

Introduction

In high-stakes domains like epidemiology or energy planning, a forecast is only as good as its reliability. Most modern forecasting problems involve "multi-view" data: for example, predicting COVID-19 spread requires looking at historical infection rates (sequences), hospital capacities (static data), and interstate travel (networks).

The authors identify two major gaps in how current AI models handle this:

  1. Naive Integration: Most models simply concatenate or sum features from different sources. This assumes all data sources are equally reliable at all times, which isn't true (e.g., mobility data might be crucial at the start of a pandemic but noisy later on).
  2. Poor Calibration: Many models provide a "point prediction" without a sense of confidence, or their "confidence intervals" are mathematically ungrounded. A well-calibrated model should be able to say, "I am 90% sure the value will fall in this range," and be right exactly 90% of the time.

Methodology

CAMul (Calibrated and Accurate Multi-view Forecasting) uses a three-stage pipeline to turn raw, multi-modal data into a calibrated probability distribution:

High-level outline of the CAMul framework showing the integration of multiple data views through probabilistic encoders and dynamic selection.
  • Multi-view Probabilistic Encoders: The model uses specialized encoders for different data types (GRUs for sequences, Graph Convolutional Networks for relational data). Instead of a single vector, these encoders output parameters for a distribution, capturing the initial uncertainty of that specific data source.
  • View-Specific Correlation Graphs (VSCG): Borrowing from Functional Neural Processes (FNP), CAMul compares the current input to a "reference set" of training examples. It builds a graph that relates the current situation to similar historical patterns, allowing the model to leverage global knowledge about how that specific data view behaves.
  • Context-Specific Dynamic View Selection: Not all views are useful all the time. CAMul uses a cross-attention mechanism that looks at the recent history of the time series and decides which data sources to trust more right now.
  • Forecast Decoder: Finally, it aggregates these weighted, uncertainty-aware embeddings to produce a final Gaussian distribution for the forecast.

Key Insights and Dynamic Weighting

A core strength of CAMul is treating uncertainty as a first-class citizen. By modeling uncertainty within each view before combining them, the model can "discount" noisy data sources automatically. If the mobility data is conflicting or messy, its probabilistic encoder will signal high variance, and the model will rely more on other, more certain views.

Dynamic attention weights shifting during the 2018 flu season, illustrating how the model adjusts its reliance on different data views around peak activity.

The authors found that during the early stages of COVID-19, the model automatically assigned higher weights to mobility data because historical infection sequences were too short to be useful. As the pandemic progressed and reporting systems stabilized, the model shifted its weight back to the historical sequences. As shown in Figure 4, the model dynamically adjusts attention weights for different views (Default, HHS Adjacency, Month) based on the context, such as decreasing weights for certain views around peak activity periods to maintain calibration.

Key Results

The performance gains were substantial across four distinct domains (google-symptoms, covid19, power, and tweet):

  • Accuracy: CAMul outperformed baselines like DeepAR and DSSM by over 25% in RMSE (Root Mean Squared Error).
  • Probabilistic Quality: In terms of CRPS (a metric for how well a predicted distribution matches reality), CAMul showed a 24% to 28% improvement over the next best models.
  • Calibration: The model’s "Confidence Score" was significantly better, meaning its predicted uncertainty intervals were much more trustworthy than those of standard deep learning models.

Limitations and Future Work

While CAMul is powerful, the authors note a few areas for growth. The model relies on a "reference set" of training data; as datasets grow into the millions of points, selecting and maintaining an efficient reference set will become a computational challenge. Additionally, the current implementation assumes a Gaussian output distribution. Future versions could be extended to handle discrete counts or heavy-tailed distributions. Finally, the authors suggest that the dynamic view weights could be used as a "canary in the coal mine" for anomaly detection—if the model suddenly stops trusting a usually reliable data source, it might indicate a sensor failure or a fundamental shift in the underlying system.

Summary updated

Abstract

Probabilistic time-series forecasting enables reliable decision making across many domains. Most forecasting problems have diverse sources of data containing multiple modalities and structures. Leveraging information as well as uncertainty from these data sources for well-calibrated and accurate forecasts is an important challenging problem. Most previous work on multi-modal learning and forecasting simply aggregate intermediate representations from each data view by simple methods of summation or concatenation and do not explicitly model uncertainty for each data-view. We propose a general probabilistic multi-view forecasting framework CAMul, that can learn representations and uncertainty from diverse data sources. It integrates the knowledge and uncertainty from each data view in a dynamic context-specific manner assigning more importance to useful views to model a well-calibrated forecast distribution. We use CAMul for multiple domains with varied sources and modalities and show that CAMul outperforms other state-of-art probabilistic forecasting models by over 25\% in accuracy and calibration.
BibTeX
@inproceedings{kamarthi2022camul,
  author = {Harshavardhan Kamarthi and Lingkai Kong and Alexander Rodríguez and Chao Zhang and B. Aditya Prakash},
  title = {{CAMul: Calibrated and Accurate Multi-view Time-Series Forecasting}},
  booktitle = {The Web Conference},
  year = {2022},
  doi = {10.1145/3485447.3512037},
}

Loading related content...

Figure