TL;DR
HOPD (Historical On-Policy Distillation) introduces a novel framework for the continual learning of Multimodal Large Language Models (MLLMs) in the context of Synthetic Aperture Radar (SAR) interpretation. By archiving source-specific LoRA states as persistent "teachers" and using on-policy distillation with asymmetric visual views, the method effectively prevents catastrophic forgetting even when historical data is extremely scarce.
What Problem Does This Paper Address?
Synthetic Aperture Radar (SAR) is vital for all-weather maritime and infrastructure monitoring, but interpretation models face a "moving target" problem. As new data arrives from different sensors or geographic regions, models must adapt without losing what they previously learned.
This is particularly difficult in SAR for three reasons:
- Acquisition & Geographic Shifts: Changes in radar frequency, polarization, or viewing geometry alter how objects appear.
- Regional Category Bias: A new dataset might be dominated by one category (e.g., ships), which can "overwrite" the model's knowledge of rarer categories (e.g., aircraft) seen in previous sources.
- Data Scarcity: Access to historical imagery is often limited, making standard "replay" techniques (re-training on old data) less effective.
Methods
The authors propose Historical On-Policy Distillation (HOPD), a two-stage recursive process that turns the learning history itself into a form of supervision.
- Stage A: Source Acquisition: The model performs standard Supervised Fine-Tuning (SFT) on the new data source using LoRA adapters. Once training is complete, this specific adapter state is frozen and archived as a "source-acquisition teacher."
- Stage B: Historical Consolidation: To prevent forgetting, the current "student" model is distilled from the bank of archived teachers.
- On-Policy Distillation: Instead of just looking at fixed historical answers, the student generates its own response "prefixes." The teacher then evaluates these student-generated contexts, providing "soft targets" that help the student maintain historical behavior even as its internal context changes.
- Cross-View Visual Distillation: To make the most of sparse replay data, the student is shown augmented (gamma-transformed) versions of old images, while the teacher evaluates the original, clean images. This forces the student to learn robust features while the teacher provides a stable reference.
Datasets
The authors establish SARVQA-CL, a new benchmark for source-incremental continual learning in SAR. It is adapted from the high-quality SARFact dataset and includes:
- 356,624 training QA pairs and 5,988 evaluation pairs.
- 9 constituent datasets (e.g., OGSOD, HRSID, MSAR) acting as sequential sources.
- 7 diverse tasks: Captioning, counting, presence detection, classification, object detection, referring expressions, and spatial relations.
Key Insights and Ideas
- LoRA States as Teachers: The core "aha moment" is realizing that the intermediate states of the model during the learning process are valuable. By freezing the model right after it masters a new source, you capture its "peak" performance for that specific domain before later updates degrade it.
- Addressing the Training-Generation Mismatch: Standard distillation often fails because the teacher and student are evaluated on different text distributions. By using "on-policy" distillation (querying the teacher on the student's own generated prefixes), the supervision becomes much more relevant to how the model actually behaves during inference.
- Asymmetric Views for Sparse Replay: When you only have one or two images from a past source, you need to maximize their utility. Using different "views" for the teacher and student acts as a powerful regularizer, preventing the model from simply memorizing specific pixel patterns.
Key Results
HOPD sets a new state-of-the-art on the SARVQA-CL benchmark:
- Superior Retention: It achieved a final mean score (MFN) of 54.51, outperforming the strongest baseline (MR-LoRA) by 2.96 points and standard Replay-SFT by 7.45 points.
- Efficiency: HOPD with only one image per source achieved performance comparable to Replay-SFT using four images per source, demonstrating its effectiveness in data-constrained environments.
- Consistency: The performance gains were consistent across different source orders and across all seven evaluated tasks, particularly in resisting "category intrusion" from dominant classes.
Limitations and Future Work
- Archive Growth: Currently, the teacher archive grows linearly with the number of sources. For very long sequences of updates, the authors suggest that selective archive compression or pruning will be necessary.
- Simplified Augmentations: The current visual augmentations (gamma transforms) are relatively simple. Future work could involve more complex modeling of radar physics, such as simulating changes in incidence angles or polarimetric properties.
- Complex Data Types: Extending the framework to handle native complex-valued SAR data (rather than just rendered intensity images) remains an open challenge.
Summary updated