Nanoscale defect inspection is slow because a scanning electron microscope (SEM) builds an image one pixel at a time, and today it gives every pixel the same exposure time. A clean stretch of resist lines gets as many electrons as an ambiguous spot that might be a nanometre-scale break. We propose a two-pass scheme. The microscope first takes a fast, low-exposure scan of the whole field. A learned model then predicts, for each region, how likely its defect decision is to change if that region were imaged at full exposure. Rescanning time goes only to the regions where the model expects the answer to flip. The goal is to keep the defect-detection accuracy of a full slow scan while spending a fraction of the beam time, and so reduce the inspection lag that stops nanofabrication from scaling.
This matters most for the parallel, high-throughput patterning methods that are meant to replace slow direct electron-beam writing. Examples include nanoimprint replication, which stamps a pattern onto a whole substrate at once (High-Performance, High-Index-Contrast Chalcogenide Glass Photonics on Silicon and Unconventional Non-planar Substrates), and electron caustic lithography, which projects patterns with an electron mirror instead of scanning them (Electron caustic lithography). These methods make features faster, but they are still checked afterwards with offline SEM. Inspection therefore becomes the step that sets the pace.
Core Claim
The new ingredient is the signal used to decide where to rescan. Existing uncertainty-guided methods score regions by total predictive uncertainty and then refine the most uncertain ones. The clearest example is Boosting the Dual-Stream Architecture in Ultra-High Resolution Segmentation with Resolution-Biased Uncertainty Estimation, which keeps only the most uncertain patches for a high-resolution branch and reports a 517x speedup over CascadePSP on aerial imagery. In SEM inspection, total uncertainty mixes two different things:
- Shot noise. This is graininess from counting too few electrons. More dwell time removes it.
- Structural ambiguity. This comes from features that remain hard to classify even at full dose, or from defect types the model has never seen. More dwell time does not help.
Ordinary photographs do not have this split, because their noise cannot be bought down per pixel. In an SEM it can. The file itself flags the problem of separating noise-driven from structure-driven uncertainty as open (Learning Heterogeneous Degradation Representation for Real-World Super-Resolution).
We claim that ranking regions by dose-resolvable uncertainty gives a better accuracy-versus-dose curve than ranking by total entropy, by anomaly score, or by denoising followed by detection. We define dose-resolvable uncertainty as the expected change in the detector's output between the low-dose and full-dose views. Concretely, for a patch observed at low dose $d$ as $x_d$, with full-dose counterpart $x_D$ and detector output $f(\cdot)$, the rescan score is
$$ s(x_d) = \mathbb{E}\left[\,\lVert f(x_D) - f(x_d) \rVert \;\middle|\; x_d\,\right], $$
and rescans are assigned greedily by $s$ divided by the extra dose each rescan costs, until the beam-time budget is used up.
A small head predicts $s$ directly. It plays the same role as the gating module in Gate to the Vessel: Residual Experts Restore What SAM Overlooks, which predicts where a frozen backbone will be wrong and routes only those regions to small correction networks. We reuse that design. The difference is that our predicted "error map" drives the physical acquisition step, not a software refinement step.
Research Program
Phase one: simulated low-dose data. We build a dose-controllable training and evaluation set from existing SEM data by simulating shorter dwell times. Each available image is treated as the full-dose reference. Lower doses are made by thinning electron counts (binomial thinning of estimated per-pixel counts), with a calibrated detector-noise term added. This approach has a limit: the available images already carry noise, so we can only simulate doses lower than the original, never higher. The evaluation therefore asks how close each method gets to the full-dose result, which is the comparison that matters in practice.
Phase two: the models. We train the detector across a range of simulated doses so it behaves sensibly on fast scans. On paired low- and full-dose views, we train the score head to regress the output change in the equation above. We then wrap both in a two-pass acquisition loop that runs entirely in simulation.
Phase three: the nanoimprint setting. Here the scalability direction enters. Stamp wear and trapped particles produce defects that recur at the same stamp coordinates across replicas. A defect-location prior accumulated over earlier replicas can therefore be added to $s$. We test whether this lowers the dose needed per replica as the run grows.
Phase four: hardware check, if access allows. A small-scale test on a microscope with programmable scan control would confirm that the simulated savings survive two real effects that simulation omits: beam settling time and charging.
There is a practical reason beyond speed for fab use. In photoresist, electron exposure itself shrinks and damages the lines being measured, so lower total dose also means less damage to the measured features.
Baselines
Full-dose uniform scan with a strong detector (the ceiling). The reference detector is the RetinaNet ensemble with U-Net denoising from Deep Learning-Based Defect Classification and Detection in SEM Images. On the 32 nm pitch resist dataset it reports 81.6% mAP overall, 67.5% AP on micro-bridges, and 52.0% AP on probable nano-gaps. We retrain it on our splits and treat its full-dose accuracy as the ceiling. Every adaptive method is judged by how much of this accuracy it keeps at a given fraction of the dose.
Uniform low-dose scan (do nothing adaptive). This is the same detector, trained on simulated fast scans, with no rescanning. It sets the floor.
Low-dose scan plus learned denoising. This is the obvious engineering answer. We use the same U-Net denoising stage as Dey et al., given the same training data.
Top-K rescanning by predictive entropy. This is the RUE-style selector from Qin et al. adapted to SEM patches, using their combination of entropy and a feature-residual measure of lost detail. It is the strongest existing uncertainty-guided method in the file. Note that no reported result found on Lacuna for RUE on any microscopy or defect dataset; its numbers (79.0% mIoU on Inria Aerial, 57 FPS on DeepGlobe) come from aerial segmentation.
Rescanning ranked by anomaly score. The scorer is the few-shot SEM detector SEM-CLIP, which reports 98.6% pixel-level AUROC and 83.8% F1-max on 55 nm CMOS SEM images with 10 labelled examples per class.
Random rescanning and an oracle. Random rescanning at matched dose is a sanity check. The oracle rescans exactly the patches whose full-dose prediction differs from the low-dose prediction, giving an upper bound on what any selector can achieve.
Fairness rules. All selectors share the same detector, the same simulated dose model, the same total electron budget (counted in rescanned pixel-dwell units), and the same hyperparameter search budget. Only the selection rule differs.
The decisive ablation. It swaps our dose-resolvable score $s$ for total entropy while holding everything else fixed. This isolates the key ingredient.
What counts as winning. Our method beats the field if it reaches within one mAP point of the full-dose ceiling at a lower total dose than every baseline. It must also beat entropy top-K on nano-gap and micro-bridge AP at equal dose, since those are the classes where noise and genuine ambiguity are most tangled.
Why not just…?
Why not just denoise the fast scan and run the detector on that? This is the most serious threat. Dey et al. already rely on denoising to cope with stochastic noise. A denoiser, however, cannot recover electrons that were never collected. Where a sub-nanometre bridge sits at the noise floor, the denoiser has to guess, and its errors are confident ones. Our experiment compares denoise-only against denoise-plus-selective-rescan at equal total dose. If denoising alone matches the ceiling at the low dose, the adaptive scheme is unnecessary and we will report that. We expect the gap to show up specifically on nano-gap and micro-bridge AP.
Why not just rescan where predictive entropy is highest, as RUE does? Entropy is also high where the structure is genuinely ambiguous or out of distribution. Spending dose there is wasted, because extra dose will not resolve it. The ablation described above answers this directly. If entropy and $s$ produce the same curves, the core claim fails.
Why not just rescan where the anomaly score is high? Anomaly scores flag regions that look unusual. They miss defects that look normal at low dose, which are exactly the faint breaks that matter. They also spend dose re-confirming obvious defects. The comparison with SEM-CLIP-ranked rescanning measures this.
Why not just use bigger GPUs or larger backbones? In this setting computation is not the bottleneck; beam time is. The systematic review Automated Semiconductor Defect Inspection in Scanning Electron Microscope Images: a Systematic Review reports throughput of roughly 763 defects per hour under acquisition-limited workflows. A faster model leaves the scan time unchanged.
Why not just sample a subset of fields statistically? Sampling misses localized defects by design, as the in-situ inspection literature in the file notes. Recurring stamp defects in nanoimprint are exactly the localized, positional failures that sampling skips. The replica-prior experiment tests this.
Proposed Experiments
Main result. We report accuracy-versus-dose curves on the 32 nm pitch resist SEM dataset from Dey et al., at simulated dose fractions from 5% to 50% plus selective rescans. The metrics are per-class AP and mAP, scored against full-dose ground truth.
Generalization check. We repeat the experiment on the 55 nm CMOS SEM set from SEM-CLIP in the 10-shot regime. This tests whether the score head transfers when only a few defect examples per class are labelled.
Decisive checks. Three results would decide the question:
- Does $s$ beat total entropy at matched dose on nano-gap AP?
- Does the method beat denoise-only at matched dose?
- Does the selector's rescan set overlap the oracle set more than entropy's set does? We measure this as recall of the patches whose decision flips.
Nanoimprint extension. Ground-truth data for repeated imprint replicas is not in the file, so we simulate it. We tile SEM patches into synthetic replica sequences and plant persistent defects at fixed stamp coordinates. The measure is how fast the per-replica dose falls once the recurrence prior is added. This is explicitly a simulation result, to be confirmed on real imprint runs.
Hardware validation, if access allows. A real-microscope run would report wall-clock time, not just dose. Beam settling and repositioning overheads could erase the savings when the rescan regions are scattered across the field. Measuring that overhead is the honest last test.
References
- High‐Performance, High‐Index‐Contrast Chalcogenide Glass Photonics on Silicon and Unconventional Non‐planar Substrates
- Electron caustic lithography
- Boosting the Dual-Stream Architecture in Ultra-High Resolution Segmentation with Resolution-Biased Uncertainty Estimation
- Learning Heterogeneous Degradation Representation for Real-World Super-Resolution
- Gate to the Vessel: Residual Experts Restore What SAM Overlooks
- Deep Learning-Based Defect Classification and Detection in SEM Images
- SEM-CLIP: Precise Few-Shot Learning for Nanoscale Defect Detection in Scanning Electron Microscope Image
- Automated Semiconductor Defect Inspection in Scanning Electron Microscope Images: a Systematic Review