TAMMs: Temporal-Aware Multimodal Model for Satellite Image Change Understanding and Forecasting

Zhongbin Guo, Yuhao Wang, Ping Jian, Xinyue Chen, Wei Peng, Ertai E

Introduction

Analyzing Satellite Image Time Series (SITS) is essential for tracking urban development, agricultural cycles, and environmental changes. However, existing research has traditionally been divided into two separate tasks: Temporal Change Description (TCD), which focuses on describing past events, and Future Satellite Image Forecasting (FSIF), which attempts to predict future visual states.

This disjointed approach often leads to forecasting models that "hallucinate" illogical changes because they lack a deep semantic understanding of historical trends. For instance, a model might predict a forest suddenly becoming a parking lot without any intervening construction phase. TAMMs (Temporal-Aware Multimodal Model) addresses this by providing the first unified framework that jointly performs both understanding and forecasting.

Comparison of prior work limitations versus the unified TAMMs framework.

The TAMMs Framework

TAMMs is built on a single architecture that integrates a Multimodal Large Language Model (MLLM) with a diffusion-based generator. The core philosophy is that high-level reasoning should guide low-level image synthesis.

Key Technical Innovations

  1. Temporal Adaptation Modules (TAM): Rather than full-model finetuning, TAM "reprograms" a frozen MLLM. It utilizes a Physical Time Encoder (PTE) to inject learnable tokens representing exact time intervals between images. This allows the model to distinguish between seasonal fluctuations and permanent structural changes over irregular time gaps.
  2. Semantic-Fused Control Injection (SFCI): This module acts as the bridge between reasoning and generation. It translates the MLLM’s abstract semantic features into multi-scale control signals for a diffusion U-Net. It employs a dual-path approach: a Structural Path for low-level visual dynamics and a Semantic Path for high-level guidance.
  3. Temporal Consistency Score (TCS): To move beyond standard pixel-wise metrics like PSNR, the authors introduced TCS. This metric uses change detection masks to quantify how well a predicted future aligns with the spatial and magnitude trends established in the past.
The TAMMs architecture, showing the flow from satellite time series through temporal encoding to joint change description and future prediction.

Experimental Setup and Results

The model was trained on a curated version of the fMoW (Functional Map of the World) dataset, containing over 37,000 multi-temporal sequences. For evaluation, the authors established a new benchmark of 150 long-term sequences (5-10 years) with expert human annotations.

TAMMs significantly outperformed state-of-the-art specialist models in both domains:

  • Understanding: Achieved a CIDEr-D score of 0.9030, surpassing the previous best specialized model, TEOChat (0.8267).
  • Forecasting: Demonstrated superior temporal consistency with a TCS of 0.9690, nearly 20 points higher than DiffusionSat (0.7624).

Qualitative analysis confirms that TAMMs can identify subtle details, such as specific parking lot expansions or changes in vegetation density, which baseline models often overlook or misinterpret.

Discussion and Future Work

While TAMMs represents a major step toward unified temporal reasoning, it faces limitations in predicting rare, abrupt events like natural disasters that lack clear historical precursors. Additionally, the reliance on a large MLLM backbone (DeepSeek-VL2) presents challenges for real-time computational efficiency. Future research will focus on model compression and integrating Bayesian approaches for uncertainty quantification, allowing the model to provide a range of possible future scenarios rather than a single forecast.

Summary updated

BibTeX
@inproceedings{guo2026tamms,
  author = {Zhongbin Guo and Yuhao Wang and Ping Jian and Xinyue Chen and Wei Peng and Ertai E},
  title = {{TAMMs: Temporal-Aware Multimodal Model for Satellite Image Change Understanding and Forecasting}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=PXDFTeIqMd},
}
Figure