When a segmentation model trained on synthetic driving scenes is deployed on real streets, many of its worst mistakes are confident ones. An entire patch of road is labeled "sidewalk" with 0.9 probability, or a truck becomes "building." Active learning methods for this setting pick target regions for a human to label. They rank regions by how uncertain the model is, typically using softmax entropy, which measures how spread out the predicted class probabilities are. Entropy cannot see a confidently wrong region. It also fires at every object boundary, including boundaries the model already gets right. This proposal uses a signal that does not depend on the model's confidence. Each pixel's prediction is compared against the predictions of the pixels it ought to agree with, meaning pixels in the same low-level image region found from color and edges. A pixel that disagrees with its own region is suspicious however confident it is. A legitimate class boundary does not trigger the signal, because each side of the boundary is internally consistent.
The same signal does two jobs. Regions where the model agrees with itself are given automatic training labels from the model's own predictions, called pseudo-labels. Regions where the model contradicts itself are sent to the human annotator. Regions that are chaotic throughout, where even the neighbors disagree with each other, are held back from both until we know whether they are learnable. The question is whether this "self-disagreement" score buys more target-domain accuracy per annotator click than the entropy-based region scores used in current active domain adaptation. A second question is whether the score transfers unchanged to a very different spatial domain, hyperspectral change detection.
Core Claim
The spatial consistency check used in semi-supervised segmentation can serve as an acquisition score for active domain adaptation. It finds a class of errors that entropy-based acquisition misses: confident mistakes caused by domain shift that contradict the surrounding image region. The neighborhood must be defined from the image itself rather than from a fixed window or from the model's own features. Under that condition, the score separates true boundaries, which need no query, from model errors, which do. We expect it to beat entropy-times-impurity region selection at equal click budgets. We also expect it to raise Boundary Intersection-over-Union (Boundary IoU, overlap measured only in a thin band around object contours), the metric that neighborhood smoothing is known to hurt.
Relation to Prior Work
Three works define what this proposal must differ from.
RIPU selects regions for synthetic-to-real active adaptation by multiplying two quantities. The first is "region impurity," the entropy of predicted classes inside a fixed k×k square window. The second is mean pixel entropy in that window. Impurity is high wherever several classes meet, so RIPU spends queries on every true boundary. Entropy also misses confident errors. Our score instead measures disagreement between a pixel and its own image region. That disagreement is near zero at a correctly predicted boundary and high for a confidently mislabeled blob.
Gao et al. replace square windows with superpixels, small groups of adjacent pixels with similar appearance. They merge superpixels across low-entropy areas using feature affinity and select by coverage and by source/target model divergence. Their merging depends on the adapted model's features, which they themselves flag as unreliable under severe early shift. Their uncertainty is still softmax entropy. We keep their click-per-superpixel annotation unit, so costs are comparable, but derive regions from raw image cues and score them by self-disagreement.
MADA selects whole images far from clusters of source-domain features, with no spatial reasoning. It serves as a weaker image-level reference.
The consistency mechanism comes from semi-supervised work. S4MC boosts a pixel's pseudo-label confidence when its N×N neighbors support the same class, and reports that this lowers Boundary IoU. TPLD densifies pseudo-labels by sliding-window voting. RRN uses patch discriminators to drop implausible regions. All three filter or smooth, and none routes the regions it rejects to a human. That is the gap we close: boundary-critical discordant regions go to the annotator rather than being smoothed or discarded.
Two papers motivate the design. Huang et al. show that geometric disagreement between predictions exposes coordinate errors better than softmax confidence. Our score is a single-model, spatial analogue of that observation. Peng et al. show that querying the most uncertain trajectory samples picks unlearnable noise, and that median uncertainty works better. That result constrains us: we do not assume the chaotic regions are informative, and we test them as a separate tier.
MCS⁴CD already chains a circular neighborhood check with margin-based active learning in hyperspectral change detection, but with the two stages kept separate. That makes it the natural testbed for whether one disagreement score can replace both stages.
Research Program
For a target image, we first compute regions from the raw image with a standard superpixel algorithm. These regions do not depend on the segmentation model, which is the point. For pixel $i$ with predicted class distribution $p_i$, let $R(i)$ be its region excluding $i$, with appearance-similarity weights $w_{ij}$. The region consensus is
$$q_i = \frac{\sum_{j \in R(i)} w_{ij}\, p_j}{\sum_{j \in R(i)} w_{ij}},$$
and the self-disagreement of pixel $i$ is $d_i = 1 - p_i(\arg\max_c q_i(c))$. This is one minus the probability pixel $i$ assigns to the class its region favors. The score is high for a confident pixel that contradicts its region.
We also compute region incoherence, $h_R = H(\bar q_R) - \overline{H(p_j)}_{j \in R}$. Here $H$ is entropy, $\bar q_R$ is the region's average prediction, and the second term is the average per-pixel entropy. $h_R$ is large when individual pixels are each fairly confident but disagree with one another, which is speckle.
These quantities define three routes. Regions with low mean $d$ receive pseudo-labels from $q$. Regions with high mean $d$ and moderate $h_R$ are query candidates. High-$h_R$ regions form a held-out tier whose usefulness we measure explicitly rather than assume. Queries are ranked by mean $d$ within each region. Each query is one click labeling one region, as in Gao et al. Selection is iterative over rounds, and pseudo-labels are recomputed after each retraining.
The analysis has three parts. The first is an oracle study that measures how much of the target error mass under the source-trained and partially adapted models lies in low-entropy pixels. If little of it does, the central premise fails. The second checks how well $d$ separates errors from correct boundary pixels. The third labels a sample of high-$h_R$ regions to test whether they are learnable or noise.
Baselines
The naive reference points are a model trained only on source data and random region selection at the same click budget. On GTAV→Cityscapes, Gao et al. report 36.6% mIoU for source-only. RIPU reports a fully supervised reference of 70.2 mIoU. Any active method should move toward the supervised reference, and random selection shows how much of the gain comes from labels alone.
The pixel-level uncertainty baseline is margin sampling with a diversity heuristic, as in PixelPick. Margin sampling queries pixels where the gap between the top two class probabilities is smallest. We run it at a matched number of labeled pixels per image. No Cityscapes number for PixelPick was found on Lacuna, so we report only our rerun.
RIPU is the clearest test of our central contrast, because its impurity term also uses neighborhood structure. It reports 69.6 mIoU on GTAV→Cityscapes with 2.2% of target labels. We rerun it with our backbone and training schedule and give it the same number of labeled pixels as our method at each budget.
The strongest existing method is the superpixel approach of Gao et al.. It reports 72.5% mIoU at 640 clicks per image and 68.7% at 40 clicks per image on GTAV→Cityscapes. We use its click unit, budgets of 40 and 640 clicks, and its cross-domain mixing and consistency regularization during retraining. Holding retraining fixed means only the selection and pseudo-label routing differ. Both methods get the same hyperparameter-search budget. Beating it means higher mIoU at 40 clicks than our rerun of Gao et al. under identical retraining, with non-overlapping spread across three seeds. We will also report its published figures alongside.
The skeptic's principled alternative is region-level epistemic uncertainty from Monte Carlo dropout: the mutual information between predictions and dropout masks, averaged over the same image-derived regions. This baseline isolates whether self-disagreement adds anything beyond a standard model-internal estimate of reducible uncertainty.
Four ablations isolate our ingredients. The first replaces image-derived regions with RIPU's square window, testing whether boundary-respecting neighborhoods matter. The second derives regions from the model's features instead of the raw image, testing Gao et al.'s concern about early-shift features. The third uses $d$ for queries but keeps standard confidence-thresholded pseudo-labels, isolating the shared routing. The fourth merges the incoherent tier into the query pool, testing Peng et al.'s noise concern.
Why not just…?
Why not just use Monte Carlo dropout or an ensemble to separate reducible from irreducible uncertainty? This is the most serious threat. Mutual-information estimates are designed for exactly that separation. Under domain shift, however, every sampled model can share the same confident mistake, because they all learned the same synthetic-texture shortcut. When that happens, disagreement between dropout samples stays low on precisely the errors we target. The file notes this kind of miscalibration under shift but gives no direct measurement. The region-averaged dropout baseline and the oracle study settle the question. If dropout mutual information recovers the confidently wrong error mass as well as $d$ does, our score adds nothing beyond cost savings, and we will say so. The two signals are also cheap to combine, so a positive interaction is a possible secondary outcome.
Why not just give RIPU or the superpixel method more clicks? Gao et al. already show that click efficiency, not peak accuracy, separates methods: 72.5% at 640 clicks against RIPU's 26,215. The open question is the low-budget regime, where 40 clicks reach 68.7%. Our claim stands or falls at 40 clicks and below, and we report full accuracy-versus-clicks curves so a "just spend more" reading is visible directly.
Why not just raise the pseudo-label confidence threshold? A strict threshold admits exactly the confident errors this proposal targets. It also starves rare classes, as TPLD reports a +4.2 mIoU gain over thresholded self-training from spatial densification. We include thresholded pseudo-labeling as the routing ablation.
Why not just query regions of median uncertainty, as in trajectory forecasting? Median-entropy selection avoids noise but also cannot see confident errors, which sit at low entropy. We run it as an additional selection rule. If it matches our score, the benefit comes from avoiding noise rather than from detecting self-disagreement.
Proposed Experiments
The main benchmarks are GTAV→Cityscapes and SYNTHIA→Cityscapes, built from Playing for Data, SYNTHIA and Cityscapes. We report mIoU, Boundary IoU, and query precision, the fraction of queried pixels that were wrong before the query, at 40, 160 and 640 clicks per image. No SYNTHIA→Cityscapes numbers for RIPU or Gao et al. were found on Lacuna, so comparisons there rely on reruns.
The first decisive check is the oracle study. If less than a meaningful share of target errors lie in low-entropy pixels after the first round, the premise is weak and the project narrows to the boundary-routing claim. The second is Boundary IoU. S4MC's documented boundary loss should reverse under our routing, because boundaries go to humans rather than being smoothed. If Boundary IoU does not improve over S4MC-style window consensus, the routing argument fails.
The cross-domain applicability test uses the Hermiston and Xiong County hyperspectral change-detection datasets. There we replace MCS⁴CD's separate circular check and margin-based active learning with the single disagreement score, keeping its classical classifier and 20 labeled samples per class. The references are 97.84% overall accuracy on Hermiston and 89.19% on Xiong County. The fragmented urban Xiong County scene is where boundary-aware routing should matter most. Success there without retuning would show that the score is a general property of spatial prediction rather than a trick tuned to driving scenes. A clear failure would mark it as specific to deep segmentation models.
References
- Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation
- Efficient Active Domain Adaptation for Semantic Segmentation by Selecting Information-Rich Superpixels
- Multi-Anchor Active Domain Adaptation for Semantic Segmentation
- Semi-Supervised Semantic Segmentation via Marginal Contextual Information
- Two-Phase Pseudo Label Densification for Self-training Based Domain Adaptation
- Semi-Supervised Semantic Segmentation With Region Relevance
- Semi-Supervised 2D Human Pose Estimation Driven by Position Inconsistency Pseudo Label Correction Module
- Privacy-Preserving and Uncertainty-Aware Federated Trajectory Prediction for Connected Autonomous Vehicles
- Multilayer Cascade Screening Strategy for Semi-Supervised Change Detection in Hyperspectral Images
- All you need are a few pixels: semantic segmentation with PixelPick
- Playing for Data: Ground Truth from Computer Games
- The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes
- The Cityscapes Dataset for Semantic Urban Scene Understanding