Distribution Optimization Under Gaussian Hypothesis for Domain Adaptive Semantic Segmentation

Liang Chen, Weihua Chen, Xin Zhao, Junyan Wang, Lijun Cao, Junge Zhang

Introduction

Unsupervised Domain Adaptation (UDA) is a critical challenge in computer vision, aimed at bridging the gap between labeled source domains (e.g., synthetic data from GTA V) and unlabeled target domains (e.g., real-world Cityscapes footage). The primary obstacle is the "domain shift"—discrepancies in lighting, texture, and style that cause a model's internal feature representations to become disorganized when encountering new environments. Traditional methods often rely on global statistics or category prototypes, which may fail to capture the complex, noisy distributions inherent in real-world data.

Methodology: The CAVI Framework

This paper proposes the Class-Aware Variational Inference (CAVI) framework, grounded in the Gaussian Hypothesis. This hypothesis suggests that for a model to generalize effectively, the features of any given category (e.g., "car" or "person") should follow a multidimensional Gaussian distribution.

The framework consists of three primary components:

  1. CAVE & CAVD (Class-Aware Variational Encoder/Decoder): These modules map features into a structured Gaussian space, ensuring the feature representation is continuous and regularized rather than chaotic.
  2. GAL (Gaussian-based Alignment Loss): Instead of point-wise prototype matching, GAL aligns the entire Gaussian distribution (mean and variance) between domains using KL divergence, ensuring consistent category "understanding" across domains.
  3. GCL (Gaussian-based Contrastive Learning): This component uses learned Gaussian distributions as stable anchors. It pulls pixels of the same category toward their respective Gaussian center and pushes pixels of different categories away, sharpening decision boundaries.
Alignment under Gaussian Hypothesis

Experiments and Results

The authors evaluated the CAVI framework on two rigorous benchmarks: GTAV → Cityscapes and Synthia → Cityscapes.

On the GTAV → Cityscapes task, the proposed method achieved a state-of-the-art mIoU (mean Intersection over Union) of 77.3%, significantly outperforming previous leading methods like MIC (75.9%) and HRDA (73.8%). For the Synthia → Cityscapes task, it reached 68.9% mIoU, surpassing the previous best by 1.1 percentage points.

State-of-the-art performance on GTAV and Synthia benchmarks

Statistical validation using the Kolmogorov-Smirnov (KS) test confirmed that high-performing models naturally tend toward Gaussian feature distributions, a tendency that CAVI explicitly reinforces. Qualitative results also showed that the framework produces more continuous segmentations, reducing noise-induced misclassifications (e.g., confusing sidewalk patches with road).

Conclusion and Future Work

The CAVI framework demonstrates that enforcing a simple statistical structure—the Gaussian distribution—on the internal feature space significantly improves a model's ability to generalize across domains. While the approach introduces some architectural complexity and remains dependent on the quality of initial pseudo-labels, its performance gains are substantial. Future research may explore applying these Gaussian modeling principles to other dense prediction tasks, such as depth estimation or instance segmentation.

Summary updated

Abstract

Domain adaptive semantic segmentation aims to transfer a model, proficient in dense image classification, from a source domain to a target domain. While various transfer methods have been explored in previous studies, we argue that the modeling of categories within the model significantly affects its transferability. Building on the Gaussian Hypothesis, which posits that each category in the feature space adheres to a multidimensional Gaussian distribution, we propose a Class-Aware Variational Inference (CAVI) training method. This approach normalizes features of different categories into distinct multidimensional Gaussian distributions. To further learn domain-independent feature distributions, we optimize the feature space using a Gaussian-based alignment strategy and incorporate Gaussian-based contrastive learning. Experimental results demonstrate that our method achieves state-of-the-art performance on the GTAV → Cityscapes and Synthia → Cityscapes benchmarks.
BibTeX
@inproceedings{chen2025distribution,
  author = {Liang Chen and Weihua Chen and Xin Zhao and Junyan Wang and Lijun Cao and Junge Zhang},
  title = {{Distribution Optimization Under Gaussian Hypothesis for Domain Adaptive Semantic Segmentation}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  doi = {10.1109/wacv61041.2025.00899},
}
Figure