Microscopes drift. Over minutes to hours of acquisition, heat, vibration and stage mechanics move the specimen relative to the optics. One effect is that the plane in focus slowly slides away from the plane of interest, which is called focal plane drift, or defocus drift. Biologists now hand these images to foundation models: large networks pretrained on broad data and used "zero-shot," meaning with no training on the target task. Yet every zero-shot microscopy evaluation we know of scores these models only on curated, in-focus images. Neither Micro-Bench nor CHAMMI-75 asks how far the focus can drift before a model's answers stop being trustworthy.
We propose to measure this directly, using real through-focus image stacks rather than synthetic blur. We will then test the two cheap remedies a lab would actually reach for. The first is to flag defocused images with a simple sharpness score. The second is to deblur the image before handing it to the model. The core claim is that each model has a defocus tolerance: the axial displacement at which its zero-shot behavior degrades by a fixed margin. This tolerance is measurable, is asymmetric above versus below focus, and is not predicted by the Gaussian-blur corruptions used in standard robustness benchmarks. If that claim is wrong, and Gaussian blur predicts real-defocus failures, the benchmark reduces to existing corruption suites. The study is designed to find out which is true.
Why this gap, and why not the obvious method
The tempting idea at this intersection is a physics-anchored, test-time inversion of drift in microscopy. Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cues does this for ordinary cameras. It optimizes the scale and shift of a pretrained diffusion depth model against a lens blur model, and reaches 0.273 m RMSE versus 0.349 m for Metric3D. Porting it to microscopes looks natural.
We rejected it as the core contribution. In microscopy, a reference stack of the sample is often available. The through-focus correlation scheme of McGorty et al. already recovers axial drift to about 20 nm by comparing live images against pre-acquired above-focus and below-focus planes. It is hard to see what a generative prior would add when a known optical model and a sharp reference already exist. A skeptical reviewer would expect the classical fit to match it, and we agree.
The closest work on defocus itself is A Unified Framework for Microscopy Defocus Deblur with Multi-Pyramid Transformer and Contrastive Learning. It trains a deblurring network on natural images, transfers it zero-shot to microscopy, and reports that cell detection AP improves by nearly 20% on CataBlur. That paper restores images. It does not ask whether downstream foundation models become reliable after restoration, or whether restoration invents structure that fools them.
Micro-Bench and CHAMMI-75 supply the zero-shot evaluation practice we build on. Micro-Bench is a vision-language benchmark over 17,235 microscopy images. CHAMMI-75 pretrains MorphEm, a channel-agnostic self-supervised Vision Transformer, on 2.8 million images. Both explicitly exclude acquisition degradation.
Wang and Han motivate the claim that real defocus is not generic blur. Their supervised deblurring networks had to be trained separately for each axial distance, because the point spread function (the image a single point of light makes through the optics) changes nonlinearly with depth. If one network cannot cover all depths, a single Gaussian kernel standing in for defocus is suspect.
Research Program
The core resource is a real-defocus evaluation built on the Broad Bioimage Benchmark Collection U2OS z-stacks used by both Wang and Han and Zhang et al. These are human U2OS cells stained with Hoechst and phalloidin, imaged in 2 µm axial steps, with the in-focus plane at z = 17. Every field of view therefore exists at many known signed displacements from focus, with a sharp reference.
On these stacks we define three zero-shot probes that need no new labels:
- Self-retrieval. Does the embedding of a defocused view still retrieve its own in-focus field of view among all fields?
- Nucleus counting. Do vision-language models report nucleus counts that agree with counts from the in-focus plane?
- Embedding drift curve. How far does a model's representation move as displacement grows, plotted separately for each sign of displacement?
The second arm extends semantic tasks. We will take the Micro-Bench perception tasks and degrade them with defocus simulated two ways: isotropic Gaussian blur, and a depth-dependent point-spread-function model. We then calibrate which simulation reproduces the real-stack failure curves from BBBC006. This lets us state defocus tolerances for organelle, stain and modality questions, even though Micro-Bench itself has no z-stacks.
Models evaluated are:
- CLIP from Radford et al., a contrastive image-text model and the generic web-trained baseline.
- MorphEm, via the CHAMMI-75 repository, the microscopy-specialist self-supervised model.
- GPT-4o and the pathology model CONCH, as evaluated in Micro-Bench.
For each model and task we report the tolerance at which the probe drops by a fixed fraction of its in-focus value. We report it separately above and below focus, with bootstrap intervals over fields of view.
The third part tests remedies. The first remedy is a failure detector: we ask whether a model's own embedding displacement predicts its task errors better than a classical sharpness score. The second remedy is deblur-then-evaluate: we run the pretrained transformer of Zhang et al. before each foundation model and check whether tolerance improves or whether apparent gains are hallucinated. The hallucination check is counting consistency against the in-focus plane.
Baselines
The naive reference point is the in-focus score itself, the displacement-zero anchor. The strongest reported number there is GPT-4o on Micro-Bench, at 62.7% coarse-perception accuracy and 51.7% fine-grained accuracy (Lozano et al.). We found no reported result on Lacuna for any foundation model's accuracy under controlled defocus. That absence is the gap this study fills, so the defocused curves have no prior number to beat. What the study must show instead is a reproducible, model-dependent tolerance with narrow intervals.
The strongest competitor for the benchmark's own reason to exist is the standard corruption approach: Gaussian blur at increasing strength, applied to in-focus images. It is cheap and already familiar. We run it on exactly the same fields, models and prompts as the real stacks, with blur strengths matched to each real plane by equal loss of high-frequency energy. The benchmark "beats" it only if real-defocus model rankings or tolerances differ materially from Gaussian-blur ones. Concretely, that means a rank correlation clearly below 0.9, or a consistent asymmetry between above-focus and below-focus that Gaussian blur cannot express.
For failure prediction, the baseline is a classical focus-quality score: variance of the Laplacian, computed on the same images. The embedding-displacement detector must achieve higher area under the ROC curve for predicting per-image task errors on held-out fields to count as better.
For restoration, the strongest competitor is the transformer of Zhang et al., used off the shelf with no microscopy fine-tuning. The best supervised restoration number in the file comes from Wang and Han: 40.860 dB PSNR and 0.9612 SSIM at 10 µm displacement on this dataset. We will report the transformer's PSNR on the same planes for context. The comparison that matters, though, is foundation-model tolerance with and without deblurring, at identical displacements.
The key ablation isolates realism of the degradation. It compares Gaussian blur, depth-dependent PSF simulation, and real defocus on identical fields.
Why not just…?
Why not just use existing blur corruptions instead of real z-stacks? This is the real threat. If Gaussian blur reproduces the real-defocus curves, our first arm adds little beyond a new dataset. The Wang and Han finding that PSF behavior is nonlinear in depth gives reason to doubt this, but it is not proof. The matched-blur comparison above settles the question directly, and we will report a negative result if the rankings agree.
Why not just filter out defocused images with a sharpness score? For quality control in a single lab this may be enough. However, a sharpness score says nothing about which model tolerates how much drift, and one threshold cannot serve models whose tolerances differ. We test whether it predicts per-image errors as well as the model's own embedding displacement. If it does, we will recommend it as the practical detector, and the contribution narrows to the tolerance measurements.
Why not just deblur first? Zhang et al. show restoration helps a detector, but a restorer trained on natural images can sharpen edges that were never there. The counting-consistency check against the true in-focus plane tells us whether deblurring restores the foundation models' answers or only their confidence.
Why not invert the drift with a pretrained prior, as Marigold does? When a reference stack exists, the through-focus correlation of McGorty et al. already estimates axial drift. We therefore leave inversion out of scope and do not claim foundation priors help there.
Decisive Checks
The study succeeds if three things hold. First, tolerances differ across models with non-overlapping intervals; a plausible example would be MorphEm tolerating more drift than CLIP. Second, above-focus and below-focus tolerances differ for at least one model family. Third, real-defocus rankings diverge from matched Gaussian blur.
It fails informatively under two outcomes. If Gaussian blur predicts everything, existing corruption suites already cover focal drift. If deblurring raises agreement on semantic questions while counting consistency falls, deblur-then-evaluate is unsafe for quantitative biology.
References
- Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cues
- Active microscope stabilization in three dimensions using image correlation
- A Unified Framework for Microscopy Defocus Deblur with Multi-Pyramid Transformer and Contrastive Learning
- Micro-Bench: A Microscopy Benchmark for Vision-Language Understanding
- CHAMMI-75: Pre-training multi-channel models with heterogeneous microscopy images
- Defocus Deblur Microscopy via Head-to-Tail Cross-Scale Fusion
- Learning Transferable Visual Models From Natural Language Supervision