GIFT: Representation Geometry Matters for Single-Domain Generalized Object Detection

Yin Zhang, Yaoyue Zheng, Yongqiang Zhang, Bogdan Raducanu, Dan LiarXiv 2026 · arXiv

TL;DR

GIFT (Geometric-Invariance Fine-Tuning) is a novel parameter-efficient fine-tuning strategy that preserves the intrinsic geometric structure of Vision Foundation Models (VFMs) to improve domain generalization. By using learnable Householder reflections to maintain representation geometry, GIFT achieves state-of-the-art results in single-domain generalized object detection, outperforming existing methods by up to 7.9% mPC with less than 1% additional trainable parameters.

What Problem Does This Paper Address?

The goal of Single-Domain Generalized Object Detection (S-DGOD) is to train a model on one "source" domain (e.g., sunny daytime driving) so that it works reliably on "unseen" target domains (e.g., foggy, rainy, or nighttime conditions).

While large-scale Vision Foundation Models (VFMs) like DINOv2 have incredible "innate" generalization abilities, standard fine-tuning methods often ruin them. When we adapt these models to a specific task using conventional techniques, the model tends to overfit to the specific statistics of the source domain. The authors discover that this happens because unconstrained fine-tuning disrupts the geometric structure of the pre-trained representations—the subtle angular and structural relationships between features that encode universal semantics. When this geometry is distorted, the model develops "spurious responses" to domain-specific noise, leading to poor performance in new environments.

Figure 1: (a) The pre-trained VFM exhibits an intrinsically stable geometry, while (b) unconstrained fine-tuning [22] often disrupts this structure, degrading generalization. In contrast, (c) by preserving geometric structure during fine-tuning, GIFT effectively improves generalization across unseen domains (DF., DR., NR., and NS.) in the DWD dataset [51]. More results are provided in Section 4.4.

Methods

The authors propose Geometric-Invariance Fine-Tuning (GIFT), which consists of two main components:

  1. Geometric Preservation (GP): Instead of directly updating the weights ($W$) of the VFM's attention layers, GIFT introduces a learnable rotation matrix ($R$). This matrix is constructed as a product of $r$ Householder reflections. Because Householder reflections are orthogonal transformations, they are "isometries"—they preserve the lengths of vectors and the angles between them. This ensures the weight space is "rotated" to adapt to the task without "stretching" or "distorting" the underlying geometric relationships of the features.
  2. Channel-Level Scaling (CS): To complement the geometric preservation, a lightweight diagonal scaling matrix is added. This allows the model to selectively emphasize task-relevant feature channels that are stable across different domains, facilitating better task-specific adaptation without breaking the global geometric structure.

GIFT is highly efficient, only optimizing the multi-head attention parameters within the transformer blocks and introducing a negligible number of new parameters.

Figure 3: An overview of our proposed method. It is composed of two key components: (a) Fine- Tuning with Geometric Preservation (Section 3.2) and (b) Channel-Level Scaling (Section 3.3). Notably, we optimize only the multi-head attention parameters within each block. The updated weights, Wnew, are obtained by adding the scaled residual and channel-wise adjustment to the original weights W .

Datasets

The model was evaluated on two major S-DGOD benchmarks:

  • Diverse Weather Dataset (DWD): Uses "Daytime-Sunny" as the source domain and evaluates on four unseen adverse weather conditions: Daytime-Foggy, Dusk-Rainy, Night-Rainy, and Night-Sunny.
  • Cityscapes-C: Uses the clean Cityscapes dataset as the source and evaluates robustness against 15 different synthetic corruptions (noise, blur, weather, digital effects) across 5 severity levels.

Key Insights and Ideas

  • Geometry is more stable than Magnitude: Statistical analysis showed that while feature magnitudes vary wildly across different domains (like fog vs. sun), the structural correlation between feature channels remains remarkably consistent.
  • Fine-tuning as Distortion: The "aha moment" is that standard Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA or VPT actually decrease the model's robustness by warping this stable geometry.
  • Preservation > Removal: Most prior work tried to explicitly "remove" domain-specific features. GIFT suggests it is more effective to simply "preserve" the high-quality, domain-invariant structures the VFM already learned during its massive pre-training.

Key Results

GIFT sets a new state-of-the-art for single-domain generalization:

  • Cityscapes-C: Achieved 43.5% mPC, a massive +7.9% improvement over the previous best method (Cauvis).
  • Diverse Weather Dataset: Achieved 57.0% mPC (with Faster R-CNN) and 59.4% mPC (with DINO detector), outperforming the previous SOTA by +2.9% and +1.9% respectively.
  • Efficiency: These gains were achieved by adding only 0.52M parameters to a DINOv2-Large backbone—less than 1% of the total parameters. It is also significantly faster at training and inference than competing "causal" adaptation methods.
  • Qualitative: Attention map visualizations show that GIFT helps the model stay focused on foreground objects (like cars and pedestrians) even in heavy fog or darkness, whereas other methods get distracted by environmental artifacts.
Figure 5: Ablation Studies on Cityscapes-C [35] dataset.

Limitations and Future Work

The authors identify a few areas for improvement:

  • Domain Alignment: If the pre-trained geometry of the VFM is fundamentally different from the target task (e.g., using an RGB-trained model for infrared imagery), the benefits of geometric preservation might be reduced.
  • Memory Constraints: While parameter-efficient, the method still requires loading large foundation models, which demands significant GPU memory.
  • Open-Vocabulary Generalization: Currently, the method is tested on fixed category sets. Extending GIFT to open-vocabulary scenarios, where the model must detect previously unseen object classes in unseen domains, is a promising future direction.

Summary updated

Abstract

Single-Domain Generalized Object Detection (S-DGOD) aims to generalize from a single source domain to unseen domains under distribution shifts. Existing efforts mainly focus on simulating unseen domains or suppressing domain-specific factors. However, they largely overlook the intrinsic generalization capability embedded in Vision Foundation Models (VFMs). In this work, we show that unconstrained fine-tuning disrupts the geometric structure of pre-trained representations, which encodes domain-invariant semantics. Motivated by this, we propose Geometric-Invariance Fine-Tuning (GIFT), a geometric-aware fine-tuning method for VFMs. Specifically, we introduce a series of learnable Householder reflections to the pre-trained weights to encourage geometry-consistent adaptation under domain shift. In addition, we propose a lightweight channel-wise scaling to facilitate effective task-specific adaptation. Extensive experiments demonstrate that GIFT consistently outperforms state-of-the-art (SOTA) approaches on two S-DGOD benchmarks. Notably, by introducing fewer than 1% additional trainable parameters into the frozen backbone, the proposed method achieves improvements of +7.9% and +2.9% in mPC over the previous SOTA on the Cityscapes-C and Diverse Weather Dataset, respectively. This finding highlights that preserving representation geometry is crucial for robust cross-domain generalization, establishing GIFT as a principled and efficient solution to domain shift.
BibTeX
@misc{zhang2026gift,
  author = {Yin Zhang and Yaoyue Zheng and Yongqiang Zhang and Bogdan Raducanu and Dan Li},
  title = {{GIFT: Representation Geometry Matters for Single-Domain Generalized Object Detection}},
  year = {2026},
  eprint = {2610.35374},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.35374},
}
Figure