Long-term time-series forecasting (LTSF) is a cornerstone of modern planning, from predicting energy grid loads to anticipating weather patterns. While most state-of-the-art models focus on squeezing every drop of information from raw numerical sequences, a new paper from researchers at Zhejiang University suggests we might be missing the forest for the trees—or rather, the picture for the numbers.
Their proposed model, AnchorMTS, introduces a clever way to integrate visual and textual "views" of historical data to refine numerical forecasts without letting the auxiliary data take over the driver's seat.
TL;DR
AnchorMTS improves long-term time-series forecasting by using self-derived visual and textual representations (like recurrence plots and LLM summaries) to refine a standard numerical prediction. By "anchoring" these multi-modal refinements to a primary numerical forecast and enforcing an explicit influence bound, it achieves state-of-the-art accuracy across major benchmarks while maintaining strict control over the final output.
What Problem Does This Paper Address?
Most LTSF models operate in a single numerical representation space. However, the same historical data can be viewed through different lenses: geometric (angular relations), recurrent (repeated states), and semantic (textual descriptions of trends).
Recent "multimodal" forecasters have tried to use these alternative views, but they often treat them as "peer" forecasters. This creates a central tension: while a picture of a trend might be worth a thousand data points, you don't necessarily want a vision model—which wasn't trained for forecasting—to completely override a specialized numerical model. The challenge is: how can we use rich, auxiliary evidence to improve a forecast without losing the reliability of the numerical backbone?
Methods
AnchorMTS solves this with a "Forecast-Anchored Fusion" architecture, which you can see in the overview below:
The model consists of three main stages:
- Self-Derived Multi-View Generation: From a single window of historical data, the model automatically generates four distinct views:
- Gramian Angular Fields (GAF): Encodes pairwise angular relations into a spatial pattern.
- Recurrence Plots: Highlights when the system returns to similar states.
- Joint Line Views: A standard visual rendering of the time-series trajectory.
- Semantic Summaries: A compact text description (generated by a lightweight LLM like GPT-4o-mini) summarizing trends and periodicity.
- Dual-Path Forecasting:
- Primary Path: A standard numerical backbone (e.g., a Transformer-based model) produces a "numerical anchor" forecast.
- Auxiliary Path: The visual and textual views are encoded by frozen CLIP models. These stable representations are then fused with the numerical state using cross-attention and FiLM (Feature-wise Linear Modulation) to propose a candidate refinement.
- Certified Refinement: This is the "anchor" part. Instead of just averaging the two paths, the model treats the auxiliary path as a source of residual evidence. A parameter-free projection layer ensures the final forecast stays within a 25% relative $L_2$ distance from the numerical anchor. This guarantees that the auxiliary views can only "nudge" the prediction, not replace it.
Datasets
The authors evaluated AnchorMTS on six widely used LTSF benchmarks:
- Electricity Transformer Temperature (ETTh1, ETTh2, ETTm1, ETTm2)
- Exchange Rate
- Weather
They tested across four standard prediction horizons ($H \in \{96, 192, 336, 720\}$), totaling 24 distinct experimental settings.
Key Insights and Ideas
- Complementarity is Key: The authors found that different views excel at different things. For instance, GAF views are particularly strong at capturing cross-channel diversity, while line and recurrence views are better for aggregate temporal dynamics.
- Refinement > Replacement: By formulating the problem as a bounded refinement rather than a simple fusion, the model avoids the "garbage in, garbage out" risk of unaligned multimodal data. It keeps the strengths of numerical models while gaining the "intuition" of visual/textual patterns.
- The "Safety Valve" of Projection: Analysis showed that 95.8% of the time, the auxiliary refinements were already within the "safe" 25% bound. The projection acts as a critical safety valve that truncates only the rare, extreme corrections that might otherwise degrade performance.
Key Results
AnchorMTS achieved the best overall performance among seven competitive supervised baselines (including TimePro, iTransformer, and PatchTST).
- Macro Accuracy: It reached a macro MSE of 0.336 and MAE of 0.367.
- Head-to-Head Wins: It outperformed the best-performing baseline in 18 out of 24 MSE settings.
- Consistent Gains: As shown in the figure below, the improvement over the strongest baseline (TimePro) was consistent across all six datasets, with the largest gains seen in the high-dimensional Weather dataset.
Limitations and Future Work
While AnchorMTS is powerful, the authors note a few areas for further exploration:
- Preprocessing Latency: Generating images and querying LLMs for every window adds overhead. While these can be cached during training, real-time inference would require optimized preprocessing pipelines.
- Frozen Encoder Dependency: The model relies on the pre-trained knowledge of CLIP. If a time-series dataset contains patterns entirely alien to CLIP's training data, the auxiliary views might be less effective.
- Hyperparameter Sensitivity: While the 25% influence bound ($\rho_f = 0.25$) worked well globally, future work could explore adaptive bounds that change based on the model's uncertainty.
In conclusion, AnchorMTS provides a robust blueprint for "safe" multimodal fusion in time-series, proving that we can indeed have the best of both worlds: the precision of numbers and the rich context of vision and language.
Summary updated