TimeMKG: Knowledge-Infused Causal Reasoning for Multivariate Time Series Modeling

Yifei Sun, Junming Liu, Yirong Chen, Xuefeng Yan, Ding WangarXiv 2025 · DOI · arXiv

Introduction: Overcoming Semantic Blindness

Traditional multivariate time series (MTS) modeling often treats variables as "anonymous statistical signals." To a standard deep learning model, a data column is merely a sequence of numbers. It lacks the context to distinguish between "engine temperature" and "stock price," even though these variables operate under vastly different physical or economic laws. Furthermore, standard models struggle to capture inherent causal relationships—such as "increasing throttle leads to higher RPM"—from noisy numerical data alone.

Comparison of modeling paradigms: (a) Traditional, (b) LLM-backboned, and (c) the proposed TimeMKG approach.

TimeMKG addresses this "semantic blindness" by treating MTS data as a combination of numerical signals and rich variable semantics. By leveraging Large Language Models (LLMs) to build a "Multivariate Knowledge Graph" (MKG), the framework injects domain-specific causal reasoning into the modeling process.

The TimeMKG Framework

The TimeMKG architecture consists of four primary modules designed to bridge the gap between textual domain knowledge and numerical patterns.

The detailed TimeMKG architecture, illustrating the flow from MKG construction to dual-modality encoding and cross-modality fusion.

1. Multivariate Knowledge Graph (MKG) Construction

The system utilizes an LLM (via the LightRAG framework) to extract causal triplets (e.g., Variable A $\rightarrow$ Relation $\rightarrow$ Variable B) from variable descriptions and external domain knowledge. This creates a structured map of variable interactions that serves as a reasoning prior.

2. Dual-Modality Encoder

  • Causal Prompt Branch: This branch tokenizes the KG triplets and uses a "Token2Vector" module to project high-dimensional LLM embeddings into a compact space compatible with the time series model.
  • Time Series Encoding Branch: Utilizing an "Inverted Embedding" strategy, this branch treats each variable as a "token" to explicitly capture statistical correlations between different sensors or metrics.

3. Cross-Modality Attention (CMA)

The fusion step employs numerical features as a "Query" to attend to the semantic "Keys and Values" derived from the Knowledge Graph. This aligns raw data patterns with the causal priors extracted by the LLM.

4. Inference Module

A Transformer-based decoder takes the fused representations and maps them to specific downstream tasks, such as long-term forecasting or classification.

Experimental Results

The authors evaluated TimeMKG across 16 diverse datasets spanning industrial automation, finance, healthcare, and energy.

  • State-of-the-Art Forecasting: TimeMKG achieved top performance in 38 out of 48 long-term forecasting sub-tasks. It showed a 15% average improvement in Mean Squared Error (MSE) over "LLM-backboned" models like TimeLLM, suggesting that using LLMs for semantic context is more effective than using them as the primary computational engine.
  • Enhanced Classification: In classification tasks, TimeMKG reached 71.0% accuracy, outperforming both classical methods (XGBoost at 69.7%) and deep learning models (TimesNet at 68.2%).
  • Interpretability: Visualization of attention scores confirmed that the model's internal focus aligns with the causal structure of the MKG, correctly identifying relevant load variables for target predictions.

Key Insights

TimeMKG demonstrates that LLMs are most effective in time series modeling when used as feature extractors for causal priors rather than computational backbones. By offloading reasoning to the KG and pattern matching to the time series encoder, the model achieves superior performance with high efficiency. Because the MKG is based on static variable metadata, it can be pre-computed, avoiding the latency issues typically associated with real-time LLM inference.

Limitations and Future Work

The framework's primary dependency is the availability of meaningful metadata; obfuscated headers (e.g., "Var_1") limit the semantic branch's utility. Additionally, the current MKG is static. Future research may explore "dynamic knowledge integration," allowing the KG to update as the model observes new numerical patterns that refine or contradict initial domain knowledge.

Summary updated

BibTeX
@misc{sun2025timemkg,
  author = {Yifei Sun and Junming Liu and Yirong Chen and Xuefeng Yan and Ding Wang},
  title = {{TimeMKG: Knowledge-Infused Causal Reasoning for Multivariate Time Series Modeling}},
  year = {2025},
  eprint = {2508.09630},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.09630},
}
Figure