Contrast-Adaptive Boundary Targets for Thermal Segmentation

In a thermal camera image, an object's outline is often physically blurry. Heat conducts across the object's edge, the optics blur fine detail, and around sunrise and sunset the target and its background can reach nearly the same temperature. The width of the region where you cannot tell "object" from "background" is therefore not a fixed property of the annotation. It varies along the outline: sharp where a hot engine meets cold asphalt, and many pixels wide where a pedestrian's legs meet sun-warmed pavement. Current thermal segmentation models are nevertheless trained against hard, pixel-exact masks, and every border pixel receives the same penalty. We propose to change that training target. The model is supervised with soft labels whose transition width, at each point on the outline, is set by the locally measured edge blur and contrast-to-noise ratio. The model is also taught to predict that width at test time.

The payoff is a segmenter that is sharp where the image supports sharpness and honestly uncertain where it does not. Two symptoms follow from forcing a hard answer where the physics makes it a coin flip. One is contour leakage along low-contrast margins. The other is confident hallucination of boundaries under near-equal temperatures ("thermal crossover"). Both are the failures the medical low-contrast literature documents, and the file suggests they are also what drags down thermal-only segmentation. The core test is simple. Against a carefully tuned fixed-width soft boundary and against degradation augmentation, adapting the width to local contrast should improve boundary accuracy and calibration specifically on low-contrast stretches of the outline, without costing anything on high-contrast ones.

Where This Sits

Two bodies of work motivate the idea. On the thermal side, Statistics of Infrared Images documents that infrared scenes lack high-frequency detail and often have flat, equilibrated intensity distributions. Open Dataset and Enhancement Method for Long-wave Thermal Diurnal Material Classification describes how contrast collapses as the target–background temperature difference goes to zero. Together these establish that boundary ambiguity in thermal images is physical and spatially varying, not annotation noise.

On the medical side, Bounding Maps for Universal Lesion Detection replaces hard box targets with linear center-to-border grading and reports that this beats both hard and Gaussian targets on low-contrast CT lesions. That paper supplies our starting technique. Its grading is fixed-shape, however, and it is used for detection anchors. Ours is a per-pixel segmentation target whose width is tied to measured image evidence.

The closest thermal work is Segmenting Objects in Day and Night: Edge-Conditioned CNN for Thermal Image Semantic Segmentation (EC-CNN). It injects edge maps from an edge detector trained on natural RGB images into DeepLabv3 through gated affine layers and reports 61.9% mIoU on the SODA benchmark. EC-CNN changes the input features and keeps a hard binary target, and its edge detector assumes step-like edges. We leave the architecture alone and change what the model is asked to output.

UNIP: Rethinking Pre-trained Attention Patterns for Infrared Semantic Segmentation shows that texture-biased masked-image pre-training hurts thermal segmentation. It distills local-plus-global attention into infrared backbones, gaining up to 13.5% average mIoU across SODA, MFNet-T and SCUT-Seg. UNIP is orthogonal to our proposal and serves as the backbone recipe, not a competitor to the target design.

Two medical lines act as constraints rather than methods. Parallel Res2Net-based Network with ReverseAttention for Polyp Segmentation and MISNet use reverse attention, which erases the confident interior so later layers focus on the margin. As the file notes, this presumes a decent coarse foreground estimate, and crossover removes exactly that. We include such a boundary head as a baseline rather than build on it.

Restoration-first approaches take a different route. DifIISR and HATIR sharpen the image before any downstream task, at diffusion-sampling cost, and HATIR requires video. Our approach adds no inference cost beyond one extra output channel.

Core Claim

Consider a point $p$ on a ground-truth boundary between two classes. Fit an error-function (blurred step) profile to the image intensity along the boundary normal. The fit yields a local step height $\Delta(p)$, a blur scale $\sigma_b(p)$, and residual noise $\sigma_n(p)$. Classical edge-localization analysis says the positional uncertainty of such an edge grows as noise divided by edge slope, and the slope is roughly $\Delta/\sigma_b$. This motivates a resolvable transition width

$$w(p) = \sigma_b(p)\left(1 + \lambda\,\frac{\sigma_n(p)}{\Delta(p)}\right),$$

where $\lambda$ is a single scalar tuned on validation data. For every pixel $q$ near the boundary, let $d(q)$ be the signed distance to the nearest ground-truth boundary point $p(q)$, positive inside the class. The soft label for that class is then

$$y(q) = \Phi\!\left(\frac{d(q)}{w(p(q))}\right),$$

where $\Phi$ is the standard normal cumulative distribution function. The complementary mass goes to the neighboring class, and pixels far from any boundary keep hard labels.

The segmenter is trained with soft cross-entropy against $y$. A lightweight extra head regresses $w$ inside a boundary band, so that at test time the model outputs a mask plus a per-pixel transition-width map.

The claim is that this contrast-adaptive target matters beyond generic label softening. Hard targets are mis-specified on low-contrast stretches. Any single fixed width is mis-specified in the opposite direction on high-contrast stretches, blurring edges the image clearly supports. Only an adaptive width is correct on both.

A practical caveat is that the public thermal datasets are 8-bit and pass through automatic gain control, the camera's automatic brightness scaling. $\Delta$ and $\sigma_n$ are therefore measured in display intensity, not temperature. This is acceptable because the target only needs contrast relative to noise in the image the network actually sees. It also means we claim nothing radiometric.

Research Program

The program has three parts.

The first is the target-construction pipeline. It fits erf profiles along ground-truth boundaries in SODA, MFNet-T and SCUT-Seg. It rejects fits with poor residuals, falling back to a dataset-median width, and it smooths $w$ along each contour. We will report the empirical distribution of $w$ and of contrast-to-noise per dataset. This alone is a useful measurement: it tells us how much of each benchmark's boundary length is physically ambiguous.

The second part is training. We use a UNIP-style infrared-pretrained backbone, falling back to an MAE-pretrained ViT if UNIP weights are unavailable, and a DeepLabv3 variant to connect with EC-CNN's numbers. Each network is trained with and without the adaptive target and width head, under identical schedules.

The third part is a controlled-degradation test set. We take real SODA and MFNet-T test images, whose ground truth is fixed, and apply three kinds of known damage: Gaussian blur of known scale, contrast compression toward the local background (pixel $= b + \alpha(I - b)$, with $\alpha \to 0$ simulating crossover), and additive sensor-like noise. Because the applied blur and contrast are known, we can check whether the predicted width map tracks them. This makes the width head a measurable instrument rather than a decoration. We will also check the reverse case, asking whether a network trained on hard targets hallucinates confident edges as $\alpha$ shrinks.

Baselines

The naive reference point is DeepLabv3 trained with hard cross-entropy on thermal-only data. On SODA, EC-CNN reports beating it by 4.8% mIoU, which puts it at about 57.1%. That value is derived from the EC-CNN paper's reported margin, not quoted directly. We retrain it ourselves so every comparison shares data splits and schedule.

The strongest existing thermal-only method with a reported number in the file is EC-CNN, at 61.9% mIoU on SODA per its paper. It is the most relevant competitor because it is the one prior method aimed squarely at low-contrast thermal boundaries. We will reproduce it on our splits. Beating it means exceeding its reproduced mIoU and, more importantly, its boundary F-score in the low contrast-to-noise strata. We will also apply our target on top of EC-CNN, because the two ideas are compatible, and a gain there shows additivity.

For the stronger UNIP-style backbone, the file gives relative gains but no reported absolute mIoU. We therefore report our own reproduction as the hard-target reference. That backbone with hard targets is the main within-architecture baseline.

The medical-borrowed competitor is a reverse-attention boundary head in the style of the Parallel Res2Net/PraNet line, grafted onto the same backbone. It represents the obvious "port the polyp method to thermal" idea. We expect it to be competitive at moderate contrast and fragile under simulated crossover, and the controlled set tests exactly that.

The ablation that isolates our key ingredient is a fixed-width soft target $\Phi(d/w_0)$. Here $w_0$ is swept over a grid with the same tuning budget we give $\lambda$, and it gets the same width-free training. A second ablation keeps the adaptive target but drops the width head. It tells us whether predicting ambiguity helps the mask or only adds an output. All variants share backbone, augmentation, epochs and number of hyperparameter trials.

Why not just…?

Why not just train with blur and contrast-compression augmentation? This is the most serious threat. Augmentation teaches robustness to degradation and might recover most of the mIoU gain on its own. We will not argue it away. Every model, baselines included, is trained with the same degradation augmentation. If adaptive targets add nothing on top in the low-contrast strata, the proposal's accuracy claim fails. The residual claim would then be that augmentation alone cannot give a calibrated, localized estimate of how ambiguous each boundary is, and the width-tracking test on the controlled set measures that directly.

Why not just use a single tuned soft-boundary width, or label smoothing? Bounding Maps already shows that soft beats hard for low-contrast lesions, so softness by itself is not new. Our hypothesis is specifically that one width is wrong somewhere: too wide on crisp edges, too narrow on smeared ones. The fixed-width sweep with a matched tuning budget is the decisive test. If the best fixed width ties adaptive on both high- and low-contrast strata, adaptivity is unnecessary and we will say so.

Why not just temperature-scale a hard-target model for calibration? Post-hoc temperature scaling fixes average overconfidence with one global scalar. It cannot make a model uncertain on one stretch of an outline and confident on another. We will report boundary-band calibration error for temperature-scaled hard-target models next to ours. If temperature scaling matches us within the band, the calibration part of the claim does not hold.

Why not just restore the image first, as DifIISR does? Restoration can sharpen edges that were merely blurred. When the target and background genuinely share a temperature, however, there is no signal to restore, and a generator may invent one. DifIISR also reports downstream segmentation gains only as per-class improvements on FMB, such as +7.4% truck mIoU, at diffusion-sampling cost. Compute permitting, we will run a restore-then-segment comparison on the controlled set. We expect it to help at moderate blur and to fail or hallucinate as $\alpha \to 0$.

Proposed Experiments

The evaluation runs on SODA, MFNet-T and SCUT-Seg, the thermal-only benchmarks used by UNIP and EC-CNN, plus the controlled-degradation versions of the SODA and MFNet-T test sets.

The metrics are mIoU, boundary F-score within a pixel tolerance band, and boundary IoU. Each is reported both overall and stratified into contrast-to-noise bins computed from the ground-truth profile fits. We also measure expected calibration error restricted to the boundary band. On the controlled set we measure rank correlation between predicted width and the applied blur scale and contrast factor.

Four results would settle the question. The first is a boundary F-score gain over the best fixed-width and augmentation-matched baselines in the lowest contrast bins. The second is no measurable loss in the highest bins. The third is a predicted width that grows monotonically as applied blur rises and applied contrast falls. The fourth concerns simulated crossover: as $\alpha$ shrinks, our model's boundary uncertainty should widen, while hard-target and reverse-attention models keep producing confident but wrong edges.

If the first two results hold only at the level of the tuned fixed width, we will report that the benefit comes from softening alone and not from physics-matched adaptivity.