Abstract
Neural network-based semantic segmentation has achieved remarkable results when large amounts of annotated data are available. However, such data is expensive to collect, leading to the development of Unsupervised Domain Adaptation (UDA) methods that adapt models trained on synthetic data to real-world domains. Current approaches often overlook how network architecture impacts transferability. This paper introduces AutoAdapt, a framework that performs neural architecture search (NAS) specifically for UDA. We identify an "optimization gap" in searching architectures without target labels and propose a novel metric using maximum mean discrepancy and regional weighted entropy to bridge it. Experimental results demonstrate that AutoAdapt discovers architectures that significantly enhance the performance of existing adaptation techniques.
The Architecture Gap in Domain Adaptation
In semantic segmentation, the high cost of manual labeling often necessitates training models on labeled synthetic "source" data (e.g., GTA5) and adapting them to unlabeled real-world "target" data (e.g., Cityscapes). While most UDA research focuses on loss functions or data augmentation, the role of the network architecture itself is frequently ignored.
A critical observation in this work is the lack of correlation between supervised performance and UDA performance. As shown in Figure 1, modern architectures like DeepLabV3+ and DAnet outperform the older DeepLabV2 in supervised settings but exhibit significantly lower performance when adapted to a new domain. This suggests that architectural features optimized for a specific dataset may actually hinder generalization.
The AutoAdapt Framework
Standard NAS requires a labeled validation set to evaluate candidate architectures. In UDA, the absence of target labels creates an optimization gap. AutoAdapt addresses this through a two-stage automated framework.
Stage I: Search and Build
Using an evolutionary algorithm, AutoAdapt mutates a "seed-net" (a pretrained backbone) to generate child models. The search space encompasses several critical hyperparameters and modules, including crop size, layer count, kernel sizes, dilation rates, and Squeeze-and-Excitation (SE) layers.
Stage II: Adaptation and Evaluation
Child models undergo standard UDA training (e.g., adversarial training). To rank these architectures without target labels, AutoAdapt employs a joint proxy metric:
- Maximum Mean Discrepancy (MMD): Measures the distance between source and target feature distributions.
- Regional Weighted Entropy (ReEnt): Evaluates prediction consistency. By partitioning images into regions of similar appearance and weighting entropy, this metric rewards models that produce smooth, consistent predictions in visually similar areas.
Key Insights and Results
The framework was evaluated on standard benchmarks: GTA5 → Cityscapes and SYNTHIA → Cityscapes.
- Superior Performance: On GTA5 → Cityscapes, AutoAdapt achieved a Mean Intersection over Union (mIoU) of 50.2%, surpassing the DeepLabV2 baseline (43.8%) and previous state-of-the-art methods.
- Architectural Preferences: The search consistently favored Squeeze-and-Excitation (SE) layers, which help align feature distributions by reweighting channel importance. Additionally, optimized dilation rates improved prediction smoothness without increasing computational overhead.
- Robustness: Architectures found by AutoAdapt proved more transferable than those discovered by standard NAS methods (like DARTS), which tend to overfit the source domain.
Conclusion
AutoAdapt demonstrates that architecture is a first-class citizen in domain adaptation. By bridging the optimization gap with a label-free proxy metric, the framework automates the discovery of robust, transferable networks. Future work will explore expanding the search space and developing more efficient, differentiable NAS methods tailored for unsupervised scenarios.
Summary updated