Networks that find hairline cracks, wires, and retinal vessels work best when they never shrink the image. High-resolution networks are vision backbones that keep a full-detail feature map alive through every layer instead of downsampling to 1/32 scale. The same property creates a weakness. The edge of a hard cast shadow is also a thin, high-contrast line, and a network built to preserve thin lines will preserve it just as faithfully as it preserves a crack. We propose to train a lightweight high-resolution network on pairs of images. In each pair the scene and its labels are identical, and only a physically rendered shadow differs: its darkness, its softness, its tint, and the shape of the object casting it. A loss term then requires the full-detail feature stream to stay the same across the pair. The deliberately hard case is the shadow of a thin object such as a cable, twig, or fence wire, which looks almost exactly like a crack.
The claim is narrow and testable. With ordinary augmentation and only a segmentation loss, the output layer learns to discard the shadows seen in training, but the high-resolution features still encode shadow edges. The network should therefore fail on shadow shapes and tints it has not seen. Enforcing invariance inside the high-resolution stream should close that gap at no inference cost, because nothing is added at test time. The failure mode is documented but unaddressed. Bai et al. show that an HRNet backbone lifts crack instance-segmentation AP from 21.7 to 59.3, yet their only defense against lighting is brightness and contrast jitter, and sharp shadow boundaries remain an open failure mode. TOS-Net runs a native-resolution edge stream on raw image gradients and has no way to tell a shadow edge from a physical edge.
Core Claim and Relation to Prior Work
Three lines of work each cover part of this problem. Only one combination of them has been tried, and none has been made invariant inside the high-resolution stream.
The first line motivates the proposal by showing that keeping resolution high is what makes thin structures detectable at all. In Bai et al., Mask R-CNN goes from 21.7 to 59.3 AP when HRNet replaces the standard backbone. TOS-Net beats downsampling interactive segmenters by about 30% thin-structure IoU on HRSOD. Lite-HRNet shows the parallel-stream design can be cheap: it replaces costly 1×1 convolutions with channel weighting and reaches 70.4 COCO keypoint AP at 0.7 GFLOPs. Lite-HRNet is the backbone we adopt. None of these papers models illumination beyond scalar jitter.
The second line supplies our augmentation method. Under the Shadow treats shadows as continuous opacity fields rather than binary masks. It builds image pairs in which shadow position is fixed and opacity varies, and it splits a shared backbone into position features and opacity features. It reports BER 1.32 on ISTD and 1.79 on FSD. We borrow the paired-opacity idea but reverse its purpose. That paper wants features that describe the shadow. We want the features used for the downstream structure to contain no shadow information. Its backbone also downsamples, and the authors note it struggles with shadows cast by thin occluders. Those thin-occluder shadows are exactly the hard negatives we synthesize.
Planckian jitter restricts color changes to physically plausible black-body illuminants between 3,000 K and 15,000 K, and it beats standard jitter on Flowers-102 (93.99% vs. 93.16%). We use it to tint the shadowed region. A region in shadow is lit mostly by skylight, which has a higher color temperature than direct sun, so a physical shadow both darkens and cools the surface. Color constancy errors act as global attacks on deep networks. Their effect inside a shadow region, where the illuminant changes locally, has not been studied.
The third line constrains the design. CrackCue is the closest existing work. It erases cracks with 8×8 max-pooling, refines a crack-free background with a second encoder-decoder, and subtracts that background from the input to produce an illumination-cancelled cue. It reports relative gains such as +11.5% F1 for CrackFormer on Stone331 and +32.4% ODS on low-contrast DeepCrack. Its kernel fails on structures wider than 8 pixels, and it costs a full extra network pass at inference. CrackCue cancels illumination before the backbone sees the image. We make the backbone itself insensitive to illumination and add nothing at test time. The two are complementary, but they make different bets, and our experiments compare them directly.
Research Program
Shadow renderer. We build a training-time renderer that works in linear RGB. Given a labeled image, it samples a shadow-casting shape and produces an opacity field $\alpha(p) \in [0,1]$ over pixels $p$. The penumbra width is drawn from a range that runs from razor-sharp to very soft. The renderer outputs
$$x'(p) = \big(1-\alpha(p)\big)\,x(p) + \alpha(p)\,\big(k\,M_T\,x(p)\big),$$
where $x$ is the original image, $k\in(0,1)$ is the attenuation of the shadowed light, and $M_T$ is the diagonal Von Kries transform for a Planckian illuminant of temperature $T$. This is the same channel-wise color-adaptation operator that Planckian jitter uses, here applied only inside the shadow. Casting shapes come from three families: large polygons such as awnings and walls, foliage-like blobs that overlap from several light directions, and thin curvilinear occluders of 1–6 pixel width that mimic cracks. The labels for $x$ and $x'$ are identical, so a thin-occluder shadow is an explicit hard negative.
Invariance objective. Let $f_s(\cdot)$ be the feature map of resolution stream $s$ in Lite-HRNet, L2-normalized per pixel. We train with
$$\mathcal{L} = \mathcal{L}_{seg}(x) + \mathcal{L}_{seg}(x') + \lambda \sum_{s} w_s \,\frac{1}{|P_s|}\sum_{p \in P_s} \big(1 - \cos(f_s(x)_p, f_s(x')_p)\big),$$
where $\mathcal{L}_{seg}$ is the usual segmentation loss, $P_s$ is the set of pixels in stream $s$, $\lambda$ sets the invariance strength, and $w_s$ weights each stream. The segmentation term keeps the features from collapsing to a constant. The weights $w_s$ let us test the central hypothesis directly: invariance matters most at the highest resolution, where shadow edges survive intact.
Diagnostic. For any trained model we measure a shadow sensitivity ratio. Its numerator is the mean feature change on the highest-resolution stream caused by a rendered shadow. Its denominator is the mean feature change caused by inserting a real crack of matched contrast. Low values are good. The ratio separates two explanations for a model ignoring shadows: its features may be truly invariant, or its output layer may have only memorized which shadows to discard.
Baselines
The naive reference is Lite-HRNet trained for segmentation with standard Albumentations brightness, contrast, and hue jitter. This is the recipe of Bai et al. moved onto a lighter backbone. Beating it on shadow-stressed test images while matching it on clean images is the minimum bar.
The key ablation uses the same renderer and the same paired images but sets $\lambda=0$, so there is only segmentation loss on both images of each pair. This isolates the feature-invariance term from the shadow data. It is the most serious competitor, and the proposal counts as successful only if full training beats this ablation on held-out shadow families.
A second ablation applies invariance only to the lowest-resolution stream rather than the highest. It tests whether the location of the constraint matters.
The strongest existing method is CrackCue placed in front of the same Lite-HRNet, and also in front of U-Net to reproduce its original setting. We use the authors' 8×8 pooling and their background network. Both CrackCue arms train on CrackTree260 with the same augmentation, so that CrackCue receives the shadow renderer as well. The file reports only relative gains for CrackCue: +3.6% F1 for U-Net on Stone331, +11.5% for CrackFormer, and +32.4% ODS on low-contrast DeepCrack. No absolute Stone331 F1 is reported on Lacuna, so we will rerun it ourselves.
We also include a shadow-removal pipeline: an opacity predictor in the style of Under the Shadow, followed by removal, followed by Lite-HRNet. For context, the strongest reported thin-structure number in the file is HRNet-backed Mask R-CNN at 59.3 AP on the Bai et al. earthquake dataset.
All arms get the same training images, the same number of iterations, and the same 20-trial tuning budget for learning rate and augmentation strength. Training compute for every arm is reported. Inference GFLOPs are reported too, so that CrackCue's extra pass is visible.
Why not just…?
Why not just add the shadow augmentation and skip the feature loss? This is the real threat, and it may work in-distribution. Our hypothesis is that output-level training without a feature constraint learns to discard the specific shadow shapes and tints seen in training, while the high-resolution features keep encoding shadow edges. The decisive test holds out entire occluder families and color-temperature ranges at training time. If the $\lambda=0$ ablation matches full training on held-out families and shows a similar sensitivity ratio, the feature loss adds nothing. We would report that result and present the renderer itself as the contribution.
Why not just use CrackCue in front of a high-resolution network? Background subtraction cancels slowly varying illumination well. A sharp shadow edge, though, is thin, and the 8×8 pooling will partly erase it along with the cracks. The edge then survives the subtraction as a bright line in the cue, the same mechanism behind CrackCue's documented failure on structures wider than 8 pixels. CrackCue also doubles inference cost. The thin-occluder test split will show whether this concern holds, and CrackCue may still win on soft, global lighting changes.
Why not just remove shadows first? Removal models trained on ISTD and SRD-type data target large shadows. Under the Shadow reports trouble with thin and overlapping casters, and color-constancy pre-processing errors are known to propagate downstream. Including the removal pipeline as a baseline measures this directly.
Proposed Experiments and Decisive Checks
Following CrackCue's protocol, we train on CrackTree260 and test across domains on Stone331, DeepCrack, and CHASEDB1, scoring with F1 and ODS. Each test set gets a clean version and a shadow-stressed version. The stressed versions use occluder families and color temperatures withheld from training. On every split we report crack F1, ODS, and the shadow false-positive rate, which is the fraction of shadow-edge pixels labeled as crack.
Because synthetic shadows can flatter a model trained on synthetic shadows, we also photograph a small real paired set. Each pair shows the same concrete and stone surfaces with and without a physical caster (cables, mesh, branches), taken at several times of day. This set is the arbiter of whether results transfer to real shadows.
To show the method is not crack-specific, we test transfer to thin objects on HRSOD with $IoU_{thin}$, against TOS-Net's reported 65.1% (four clicks), with shadows rendered onto the test images. Phi-Net's structure-level tier is a further scale stress test, where Bai et al.'s Mask R-CNN with A-PANet reports 81.9% accuracy.
The proposal stands or falls on three checks. First, full training must beat the $\lambda=0$ ablation on held-out shadow families and on the real paired set, with clean-image F1 within one point. Second, the shadow sensitivity ratio of the highest-resolution stream must fall well below that of both baselines, and constraining the highest stream must matter more than constraining the lowest. Third, the method must match or beat CrackCue-plus-Lite-HRNet on thin-occluder shadows while adding no inference cost. If the first check fails, the paper becomes a study of a shadow renderer. If the third fails, the evidence favors cancelling illumination before the backbone over teaching the backbone to ignore it.
References
- End-to-end Deep Learning Methods for Automated Damage Detection in Extreme Events at Various Scales
- Deep Interactive Thin Object Selection
- Lite-HRNet: A Lightweight High-Resolution Network
- Under the Shadow: Exploiting Opacity Variation for Fine-grained Shadow Detection
- Planckian jitter: enhancing the color quality of self-supervised visual representations
- What Else Can Fool Deep Learning? Addressing Color Constancy Errors on Deep Neural Network Performance
- Coarse-to-fine crack cue for robust crack detection