Near-infrared fluorescence imaging in the second window (1000–1700 nm, "NIR-II") lets surgeons see blood vessels, lymph nodes and tumor margins millimeters to centimeters below the tissue surface. Flexible fiber-bundle endoscopes can carry this capability inside the body, but at these wavelengths light leaks between neighboring fibers. This leakage, called inter-core crosstalk, blurs the image and overlays a honeycomb pattern. Neural networks can undo much of the damage, but two problems remain. First, a network tuned for clean bench images may fail, or invent structures, when crosstalk, scatter or water absorption get worse in a real patient. Second, making it robust usually costs accuracy on the good images. Architecture search, the automated design of network wiring, could find networks that avoid this trade-off. Doing so, however, normally means training thousands of candidate networks and then attacking each one to measure its robustness.
This note proposes a cheap score for ranking untrained candidate networks by how well they will hold up under physical optical degradation. The score feeds the network clean images and images passed through a simulated fiber-bundle forward model, then measures how consistent its responses are. It is corrected for a known bias against skip connections, which encoder-decoder restoration networks cannot do without. The score is validated against a new ground-truth benchmark of a few hundred encoder-decoder networks, each trained with physics-based augmentation. The central question is deliberately narrow. Once augmentation has already been used, is there any architecture-dependent robustness left over, and can a cheap score find it?
Core Claim
We claim three things. First, among encoder-decoder restoration networks trained with the same physics-aware augmentation, the architecture still determines how much clean image quality is lost under crosstalk, scatter and 1450 nm water-absorption dropout. We expect this spread to be large enough to matter clinically, by analogy with the 20%–41% spread in mean adversarial robustness that Neural Architecture Design and Robustness: A Dataset found among equal-parameter NAS-Bench-201 cells.
Second, a zero-cost score that uses physically simulated perturbations will rank these networks better than scores that use gradient-sign perturbations or clean data only.
Third, the known skip-connection bias of robustness proxies is a measurement artifact, and a simple correction for it at initialization removes it without hurting ranking.
The second claim is not obvious. Two prior results point in opposite directions. An Evaluation of Zero-Cost Proxies found that computing proxies on perturbed inputs gave no benefit over clean inputs. CRoZe (Generalizable Lightweight Proxy for Robust NAS against Diverse Perturbations) argues that consistency between clean and perturbed inputs is essential. Both were tested only on classification with digital or single-step linear perturbations. Nonlinear, spatially structured optical degradation is exactly the case where their disagreement should be resolved.
Relation to Prior Work
The closest work is robustness-aware zero-cost search, and this proposal reuses its methods.
CRoZe scores untrained networks by the consistency of features, gradients and parameter updates under one FGSM-style perturbation of inputs and weights. It reports a Spearman correlation of about 0.97 with trained models on CIFAR and CIFAR-C. Proxy $R$ (Zero-cost Proxy for Adversarial Robustness Evaluation) combines the smallest eigenvalue of the neural tangent kernel with a finite-difference curvature estimate. It reaches a Kendall tau of 0.33 on 223 adversarially trained models, but it penalizes skip connections at initialization. An Evaluation of Zero-Cost Proxies trains a random forest on 15 proxies plus 191 graph features and reaches $R^2$ of 0.70–0.80 for joint clean-plus-robust prediction on NAS-Bench-201.
All three work on classification cells under $L_\infty$ attacks or synthetic corruption. We move the question to dense restoration under a physical forward model, where skip-heavy U-Net-like wiring is mandatory.
Two further papers motivate the trade-off. RobustART shows that convolutional networks pay a clean-accuracy cost under adversarial training, while transformers sometimes do not. Neural Architecture Design and Robustness: A Dataset finds that robust cells stack convolutions rather than using parallel skip paths. Both findings are for classification. Whether they carry over to restoration, where skips carry high-frequency detail, is part of what we test.
C2FNAS supplies the search-space design. It uses a coarse-to-fine medical search with a U-Net-shaped macro topology, and its searched model beat nnU-Net across the Medical Segmentation Decathlon. It optimizes only clean Dice and trains its coarse-stage candidates from scratch. We keep its shape priors and replace the training with our proxy.
The application and its gap come from Agentic AI-powered flexible fiber-bundle endoscopy. Its GAME pipeline combines residual diffusion, vision-language-model routing and an SSIM-based check for hallucinated structure. It reaches in vivo resolution of 0.30 mm against a 0.29 mm wide-field reference. Its networks were chosen by hand, its robustness to worsening degradation was never evaluated, and its latency is unquantified. Our target is a single compact network that fits a real-time budget. The same pipeline also provides the optical parameters (12 µm cores, 0.55 NA) and the 96,050-image paired DMD dataset that our experiments rely on.
Research Program
Forward model. We will first build a differentiable fiber-bundle degradation model with four components:
- honeycomb sampling at the stated core geometry;
- a nearest-neighbor coupling kernel whose strength increases with wavelength, representing crosstalk;
- depth-dependent blur scaling as $\lambda^{-\alpha}$, following the scattering behavior described in Water-soluble polymer brush-substituted squaraine NIR-II dye;
- signal attenuation near 1450 nm, representing the water absorption described in Cherenkov excited luminescence imaging in the NIR-II window, plus photon shot noise.
Each component has a severity knob. We will calibrate these knobs so that simulated degradations match the real paired DMD images at the corresponding wavelengths.
Ground-truth benchmark. Next we build what does not yet exist: roughly 300 encoder-decoder restoration networks sampled from a C2FNAS-style space. The space varies depth, width, skip pattern (none, long only, dense), operation type and upsampling. Every candidate is trained with an identical recipe that already includes physics augmentation at moderate severity.
Each trained network is then evaluated on three things:
- clean paired images;
- held-out higher severities of the modeled degradations;
- one degradation deliberately left out of training augmentation, namely the water-band dropout, to test whether robustness transfers to an unseen corruption.
We summarize each network with a restoration version of the harmonic robustness score from On Adversarial Robustness: A NAS perspective:
$$H(a) = \frac{2\, Q_{\text{clean}}(a)\, Q_{\text{deg}}(a)}{Q_{\text{clean}}(a) + Q_{\text{deg}}(a)}$$
Here $Q_{\text{clean}}(a)$ and $Q_{\text{deg}}(a)$ are the SSIM of architecture $a$ on clean and degraded test images. We also record inference latency on a fixed GPU.
Proxy. For an untrained architecture $a$, the proxy computes a CRoZe-style multi-level consistency between the network's responses to an image $x$ and to its forward-model degradation $T_\theta(x)$, with $\theta$ sampled across severities. Unlike CRoZe, it compares decoder feature maps at every scale rather than a single class output.
To remove the skip-connection bias, residual and skip branches are rescaled at initialization so that each path contributes equal output variance. The score is computed after a short statistics-only batch-norm pass on clean images, with no weight updates.
Search. The proxy then drives an evolutionary search under a latency ceiling. The best few candidates are trained fully.
Baselines
The naive reference point is a plain U-Net trained with the same physics augmentation, plus a random choice among benchmark architectures that satisfy the latency limit. If the searched network does not clearly beat random choice within the latency budget, the search adds nothing.
The strongest hand-designed reference is GAME's structural restoration model from Agentic AI-powered flexible fiber-bundle endoscopy. The file reports only resolution figures for it (0.30 mm FWHM in vivo). No reported PSNR/SSIM result was found on Lacuna for GAME's structural model on the DMD dataset. We will therefore retrain it on the same data split with the same augmentation, and we will report its latency alongside its quality. Beating it means matching its clean SSIM within noise while exceeding its degraded SSIM, at a fraction of its latency.
For the proxy itself, the strongest competitors are:
- CRoZe with its original FGSM perturbation;
- proxy $R$;
- the random-forest predictor of An Evaluation of Zero-Cost Proxies, trained on a 20% subset of our benchmark, which is the fairest supervised competitor;
- clean-data proxies (jacov, nwot, synflow, and EZNAS-A, which reports a Kendall tau of 0.65 on NAS-Bench-201 CIFAR-10 in EZNAS);
- parameter count and FLOPs.
On classification, the published numbers to keep in mind are CRoZe's Spearman correlation of about 0.97 on CIFAR-10/100-C and proxy $R$'s Kendall tau of 0.33 on Tiny-RobustBench. None of these has a reported result on dense optical restoration.
All proxies get the same number of input batches, the same wall-clock budget and the same evaluation on the same 300 networks. Beating them means a higher Kendall tau with $H(a)$, both overall and within the top decile, which EZNAS identifies as the hard region for proxies.
Two ablations isolate the key ingredients. The first swaps the physical perturbation for an FGSM perturbation and for clean-only computation; this directly tests the Lukasik-versus-CRoZe disagreement. The second turns the skip rescaling off.
Why not just…?
Why not just train a good U-Net with physics augmentation and skip the search? This is the real threat, and the benchmark is built to measure it. Every candidate already receives that augmentation. If trained networks differ in $H(a)$ by less than seed-to-seed noise, the answer is that architecture barely matters here, and we will report that as the finding. The NAS-Bench-201 spread in Neural Architecture Design and Robustness: A Dataset suggests otherwise, but it comes from classification under standard training. The held-out water-band test also covers the robustness overfitting that CRoZe reports for augmentation tailored to specific noise types.
Why not just use CRoZe or proxy $R$ off the shelf? Proxy $R$'s documented bias against skip connections (Zero-cost Proxy for Adversarial Robustness Evaluation) would systematically reject U-Net-like candidates. CRoZe's single linear perturbation may not reflect spatially structured crosstalk. Both are run as baselines. If CRoZe's FGSM variant ranks as well as the physics variant, we will conclude that the physical model is unnecessary for the proxy and keep it only for evaluation.
Why not just pick the architecture with the best clean quality? If clean SSIM alone predicts $H(a)$, then clean-only proxies suffice. An Evaluation of Zero-Cost Proxies suggests they do not, since robustness required graph features beyond the clean-accuracy signal. We test this directly by measuring how much of the variance in $H(a)$ that clean SSIM leaves unexplained across the benchmark.
Why not just ensemble several networks? Random ensembles gave nearly 12% more PGD accuracy in On Adversarial Robustness: A NAS perspective. We include a three-network ensemble in the comparison and report quality against latency. It only counts as a winner if it fits the real-time budget.
Decisive Checks
The first check is a go/no-go. Among augmentation-trained networks at matched latency, is the spread in degraded SSIM larger than the seed-to-seed variation? We measure seed variation by training 20 networks with three seeds each.
The second check is whether the physics proxy's Kendall tau with $H(a)$ exceeds both the FGSM and clean-only variants, with confidence intervals from bootstrapping over architectures.
The third check is whether the searched network beats the retrained GAME structural model and the augmented U-Net on degraded SSIM, with clean SSIM no worse, at real-time latency. We confirm this on held-out in vivo DMD images and on the two-photon kidney segmentation task from Three Dimensional Fluorescence Microscopy Image Synthesis and Segmentation, where the reported reference is 95.6% accuracy.
Finally, the proxy should not lose ranking quality on the classification robustness dataset of Neural Architecture Design and Robustness: A Dataset. This confirms that the skip-connection correction does not simply trade one bias for another.
References
- Neural Architecture Design and Robustness: A Dataset
- An Evaluation of Zero-Cost Proxies - from Neural Architecture Performance Prediction to Model Robustness
- Generalizable Lightweight Proxy for Robust NAS against Diverse Perturbations
- Zero-cost Proxy for Adversarial Robustness Evaluation
- Robustart: Benchmarking robustness on architecture design and training techniques
- C2FNAS: Coarse-to-Fine Neural Architecture Search for 3D Medical Image Segmentation
- Agentic AI-powered flexible fiber-bundle endoscopy for high-resolution NIR-II fluorescence imaging in vivo
- Water-soluble polymer brush-substituted squaraine NIR-II dye for efficient photothermal therapy
- Cherenkov excited luminescence imaging induced by megavolt X-ray beams in the second near-infrared window
- On Adversarial Robustness: A Neural Architecture Search perspective
- EZNAS: Evolving Zero-Cost Proxies For Neural Architecture Scoring
- Three Dimensional Fluorescence Microscopy Image Synthesis and Segmentation