When a driving model segments a foggy or night-time street, the thin things break first. A lamp pole becomes three disconnected blobs, a fence becomes dashes, and the curb line between road and sidewalk develops gaps. These breaks barely move pixel-overlap scores, but they matter for anything downstream that reasons about objects or boundaries. Two families of fixes exist, and in adverse weather they work against each other. Label-free adaptation methods teach a "student" network to agree with a "teacher" network's guesses on unlabeled target images, and they know nothing about connectivity. Topology-aware methods for semi-supervised learning do enforce connectivity, but they decide which structures are real with a fixed rule: a structure that only shows up faintly is treated as noise and erased. Fog and darkness make real thin structures faint, so this rule erases exactly the structures we most need to keep.
We propose to settle the question of which faint structures are real with evidence from a second image of the same place. The ACDC benchmark pairs every adverse-condition image with a GPS-matched clear-weather image of the same location (ACDC). The two images are not pixel-aligned, since they come from different drives, viewpoints, and traffic. Static structures still have the same topology in both: one pole is one connected piece, and a curb is one unbroken curve. Our core claim is that for loosely aligned image pairs, connectivity is the right signal to transfer from the clear image to the adverse one, and per-pixel labels are the wrong one. A faint structure in the adverse prediction should be kept and strengthened when it has a counterpart in the clear-weather prediction, and suppressed when it does not. This replaces the fixed persistence threshold of prior work with cross-condition evidence, and it needs no target labels.
Background and the gap
Some terms first. Persistent homology is a tool that sweeps a threshold over a predicted probability map and records when each connected piece or loop appears ("birth") and disappears ("death"). The gap between birth and death, called persistence, measures how robust the structure is. TopoSemiSeg and its extension (noise-aware topological consistency) use this tool in a teacher-student setup on histopathology. Features above a persistence threshold τ are matched between student and teacher, and features below τ are penalized as noise. On CRAG with 20% labels, this lowers Betti matching error from 10.900 (XNet) to 8.575. That paper motivates our design, and it also gives the assumption we reject. Its own stated limitation is that a static τ does not adapt to varying contrast, and its scans are clean microscopy, where faint really does mean spurious.
Two papers supply the methods we reuse. TopoMatch shows that matching barcodes by lifespan alone pairs features that are spatially unrelated, wrongly up to 99.4% of the time. It fixes this with induced matching, which pairs features only when they overlap in a shared comparison image. We take this spatially faithful matching and loosen it to tolerate misalignment between the two images. MIC supplies the base adaptation recipe. A student sees a target image with 70% of patches masked and learns to match an exponential-moving-average teacher on the full image. MIC reports +4.3 mIoU on Dark Zurich night-time adaptation. It is topology-agnostic, and heavy masking physically cuts thin structures. B-MIM makes the related point that context-dominated objectives neglect fine tubular detail.
The nearest prior work joins topology with label-free adaptation, but under the wrong kind of shift. Leveraging Topology for Domain Adaptive Road Segmentation adapts aerial road segmentation across geographies. It uses a skeleton head, a conformity loss, and a hand-crafted 8-neighborhood gap-filling step applied to pseudo-labels, and reports APLS gains of at least 9.8% on SpaceNet→DeepGlobe. It never meets weather, and it has no source of evidence for which gaps are real. The adverse-weather adaptation literature has the opposite blind spot. The cross-domain transformer aligns attention and reaches 56.5 mIoU on Cityscapes→ACDC, but measures nothing structural. Methods that use geometry as a weather-invariant anchor, such as surface-normal freespace adaptation, work for flat bulky regions but cannot separate a curb from the flat ground beside it. As far as the research file shows, no method uses the clear-weather twin as a source of connectivity evidence.
Core Claim
Let $S$ be the student's predicted probability map for a static thin class (pole, fence, traffic sign, or the road–sidewalk boundary) on the masked adverse image. Let $R$ be the teacher's prediction for the same class on the clear-weather twin, warped into the adverse frame by a coarse global alignment and then dilated by radius $r$ to absorb residual misalignment. We compute the persistence diagram of $S$ and induced-match its features against $R$ in the style of TopoMatch. Matching here uses overlap with dilated $R$ in place of exact overlap. A student feature $f$ has persistence $\pi(f)$. Matched features form the set $M$ and unmatched ones the set $U$. The loss is
$$ \mathcal{L}_{\text{twin}} = \sum_{f \in M} \big(\pi^{*} - \pi(f)\big)_{+}^{2} \;+\; \lambda \sum_{f \in U,\; \pi(f) < \tau} \pi(f)^{2}, $$
where $\pi^{*}$ is a target persistence (we use the matched reference feature's persistence), $(\cdot)_+$ keeps only the positive part, and $\lambda$ weights the removal term. Faint structures that have a clear-weather twin are pushed to become robust. Faint structures without a twin are removed, as in TopoSemiSeg, but only after the twin has had a chance to vouch for them.
The loss is restricted to static classes, so that cars present in one image and absent in the other do not create false evidence. It is computed mostly on connected components. Component tracking via union-find is close to linear time in 2D, which avoids the O(n³) cost that TopoMatch flags for general matching. Loops are computed only on downsampled road/sidewalk maps.
Research Program
We build on the MIC teacher-student pipeline with a transformer segmentation backbone. Training uses Cityscapes as labeled source and ACDC adverse images plus their clear-weather twins as unlabeled target. The term $\mathcal{L}_{\text{twin}}$ is added to MIC's masked consistency loss.
The analysis has three parts. First, we measure the failure we claim exists. On ACDC validation, we use ground-truth thin-class components to count how many genuine components fall below τ in fog and at night, and how many a TopoSemiSeg-style removal loss would erase. If few genuine components fall below τ, the motivation collapses and we report that.
Second, we characterize misalignment tolerance. We sweep the dilation radius $r$ and measure the matching precision of twin-gated decisions against ground truth, broken down by condition. Night images have larger twin offsets, so we expect them to need larger $r$.
Third, we test the label-efficient regime. We add 1% and 5% labeled ACDC images and ask whether twin supervision closes the gap more than the same number of additional labels does.
Baselines
The naive reference is a source-only model trained on Cityscapes and evaluated directly on ACDC. It shows how much fragmentation adverse conditions cause before any adaptation.
The strongest existing method is MIC, run in our codebase with identical backbone, iterations, crop size, and tuning budget. We found no reported result on Lacuna for MIC on Cityscapes→ACDC. The strongest Cityscapes→ACDC number in the file is 56.5 mIoU from the cross-domain transformer, which we also reproduce. Beating MIC means two things together: lower thin-class component-count error and higher thin-class clDice on fog and night, with mIoU no more than 0.5 points below MIC. A topology gain bought with a pixel-accuracy loss does not count.
Three topology-aware combinations isolate which ingredient matters. MIC plus clDice adds centerline overlap with no persistence or twin. MIC plus the TopoSemiSeg loss with a tuned static τ is the closest prior mechanism. MIC plus the 8-neighborhood refinement from the road adaptation work is the heuristic route. A further baseline uses the twin pixel-wise. It warps the clear-weather prediction into the adverse frame and uses it directly as a cross-entropy pseudo-label. We implement this ourselves, because correspondence-based pixel transfer is the obvious use of ACDC's pairs, even though no such method appears in the file.
The key ablations keep everything fixed and vary one piece. Replacing the twin gate with a per-image contrast-adaptive τ tests whether the twin adds anything beyond adaptivity. Setting $r=0$ tests whether tolerance is needed. Applying the loss to all classes, dynamic ones included, tests the static-class restriction. All methods share compute, and each receives an equal hyperparameter-search budget on a held-out split of ACDC training data.
Why not just…?
Why not just warp the clear-weather prediction and use it as a pixel pseudo-label? This is the most serious competitor. It uses the same extra information, and it is simpler. Our argument is that thin structures are where pixel transfer fails. A 3-pixel misalignment of a 4-pixel-wide pole turns a correct label into mostly wrong pixels, while the pole is still one component in both images. We test this directly by binning ACDC pairs by estimated residual offset. Our prediction is that the pixel-warp baseline loses its thin-class gains as offset grows while the twin loss does not. If pixel warping matches us across all bins, the topological framing is unnecessary, and we will say so.
Why not just make the persistence threshold adaptive to local contrast? This addresses the Gap 2 failure without needing a second image, and TopoSemiSeg itself names threshold rigidity as its limitation. Lowering τ in dark regions also keeps more genuine noise, because fog and sensor grain produce faint fragments too. Contrast alone cannot tell a faint pole from a faint smudge, while the twin can. The contrast-adaptive ablation measures exactly this as precision and recall of kept components against ground truth.
Why not just add clDice to MIC? Centerline overlap rewards continuity with a pseudo-label. Under fog, the teacher's pseudo-label is itself fragmented, so clDice teaches the student to copy the break. The MIC-plus-clDice baseline will show whether this happens. We expect gains in clear and rain conditions but little change in fog and at night.
Why not just defog or enhance the image first, or stitch gaps afterwards? Restoration cascades can blur the fine edges that thin classes depend on, as the MetaFusion discussion notes. Graph stitching in the style of DeepRoadMapper takes about 90 seconds per large tile and cannot tell a true gap from a missing structure. The 8-neighborhood refinement baseline stands in for post-hoc stitching. Our method adds nothing at inference time.
Decisive Checks
All evaluation uses the ACDC validation split, split by fog, night, rain, and snow, and excludes ACDC's invalid-mask regions. Metrics are mIoU, clDice and component-count (Betti-0) error on pole, fence, sign, and the road–sidewalk boundary, and a "genuine faint component recall" measuring how many ground-truth components with low predicted persistence survive.
The proposal succeeds if three things hold together. On fog and night, the twin loss beats both MIC and MIC plus static-τ TopoSemiSeg on component error and faint-component recall. It beats the contrast-adaptive τ ablation, which shows that the twin evidence carries the gain. And it beats pixel-warp transfer in the high-offset bins.
As a secondary check, we transfer the trained model to Dark Zurich night images, where MIC reports +4.3 mIoU. This tests whether the connectivity learned from twins carries over to a different dataset.
The idea fails if genuine thin structures rarely fall below τ under adverse conditions, or if pixel-warped twins do as well everywhere. Either outcome is a clean negative result that the community currently lacks.
References
- ACDC: The Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding
- Semi-supervised Segmentation of Histopathology Images with Noise-Aware Topological Consistency
- Topologically Faithful Image Segmentation via Induced Matching of Persistence Barcodes
- MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation
- B-MIM: Biased Masked Image Modeling for Generalizable Segmentation of Fine-Grained Anatomical Structures
- Leveraging Topology for Domain Adaptive Road Segmentation in Satellite and Aerial Imagery
- Exploring Consistency in Cross-Domain Transformer for Domain Adaptive Semantic Segmentation
- Cross-Modality Domain Adaptation for Freespace Detection: A Simple yet Effective Baseline
- MetaFusion: Infrared and Visible Image Fusion via Meta-Feature Embedding from Object Detection
- DeepRoadMapper: Extracting Road Topology from Aerial Images