Introduction: Overcoming Semantic Blindness
Traditional multivariate time series (MTS) modeling often treats variables as "anonymous statistical signals." To a standard deep learning model, a data column is merely a sequence of numbers. It lacks the context to distinguish between "engine temperature" and "stock price," even though these variables operate under vastly different physical or economic laws. Furthermore, standard models struggle to capture inherent causal relationships—such as "increasing throttle leads to higher RPM"—from noisy numerical data alone.
TimeMKG addresses this "semantic blindness" by treating MTS data as a combination of numerical signals and rich variable semantics. By leveraging Large Language Models (LLMs) to build a "Multivariate Knowledge Graph" (MKG), the framework injects domain-specific causal reasoning into the modeling process.
The TimeMKG Framework
The TimeMKG architecture consists of four primary modules designed to bridge the gap between textual domain knowledge and numerical patterns.
1. Multivariate Knowledge Graph (MKG) Construction
The system utilizes an LLM (via the LightRAG framework) to extract causal triplets (e.g., Variable A $\rightarrow$ Relation $\rightarrow$ Variable B) from variable descriptions and external domain knowledge. This creates a structured map of variable interactions that serves as a reasoning prior.
2. Dual-Modality Encoder
- Causal Prompt Branch: This branch tokenizes the KG triplets and uses a "Token2Vector" module to project high-dimensional LLM embeddings into a compact space compatible with the time series model.
- Time Series Encoding Branch: Utilizing an "Inverted Embedding" strategy, this branch treats each variable as a "token" to explicitly capture statistical correlations between different sensors or metrics.
3. Cross-Modality Attention (CMA)
The fusion step employs numerical features as a "Query" to attend to the semantic "Keys and Values" derived from the Knowledge Graph. This aligns raw data patterns with the causal priors extracted by the LLM.
4. Inference Module
A Transformer-based decoder takes the fused representations and maps them to specific downstream tasks, such as long-term forecasting or classification.
Experimental Results
The authors evaluated TimeMKG across 16 diverse datasets spanning industrial automation, finance, healthcare, and energy.
- State-of-the-Art Forecasting: TimeMKG achieved top performance in 38 out of 48 long-term forecasting sub-tasks. It showed a 15% average improvement in Mean Squared Error (MSE) over "LLM-backboned" models like TimeLLM, suggesting that using LLMs for semantic context is more effective than using them as the primary computational engine.
- Enhanced Classification: In classification tasks, TimeMKG reached 71.0% accuracy, outperforming both classical methods (XGBoost at 69.7%) and deep learning models (TimesNet at 68.2%).
- Interpretability: Visualization of attention scores confirmed that the model's internal focus aligns with the causal structure of the MKG, correctly identifying relevant load variables for target predictions.
Key Insights
TimeMKG demonstrates that LLMs are most effective in time series modeling when used as feature extractors for causal priors rather than computational backbones. By offloading reasoning to the KG and pattern matching to the time series encoder, the model achieves superior performance with high efficiency. Because the MKG is based on static variable metadata, it can be pre-computed, avoiding the latency issues typically associated with real-time LLM inference.
Limitations and Future Work
The framework's primary dependency is the availability of meaningful metadata; obfuscated headers (e.g., "Var_1") limit the semantic branch's utility. Additionally, the current MKG is static. Future research may explore "dynamic knowledge integration," allowing the KG to update as the model observes new numerical patterns that refine or contradict initial domain knowledge.
Summary updated