Fleet-Yield Triage: Designing Cheap Optics So Every Manufactured Unit Stays Safe

Clinics without radiologists or ophthalmologists increasingly rely on a cheap camera plus an AI model that sorts scans. The model clears the clearly normal ones and sends the rest to a remote specialist. That only works if the model almost never clears a sick patient. The safety number reported for these systems, the false-negative rate (the fraction of diseased cases wrongly cleared), is measured on one carefully calibrated device. A low-cost optical module made in thousands of copies does not behave like one device. Every molded lens is a little tilted, off-center, or mis-thick, so every unit blurs images slightly differently. A single factory-set decision threshold then yields a different false-negative rate on each unit. We propose to design the optics and the classifier together against a fleet-level safety target: across the distribution of manufacturing errors, 95% of units must keep their false-negative rate below a clinical ceiling, using one shared threshold and no per-unit recalibration. Subject to that, the number of normal exams cleared is made as large as possible.

The quantity being optimized is borrowed from optics, where a lens is judged by its "yield", meaning the fraction of manufactured copies that meet spec. We carry that idea into triage. A unit "meets spec" if it stays clinically safe, not if it produces a sharp picture. The work tests whether shaping the physical lens for this triage-specific yield buys anything beyond the obvious cheaper fix, which is to train the classifier on physically simulated blur behind an ordinary lens.

Core Claim

Tolerance-aware optical design today optimizes image fidelity. Tolerance-Aware Deep Optics builds a differentiable ray tracer, a lens simulator whose outputs can be backpropagated through. It samples 64 random assembly errors per training step (tilt, decentration, center-thickness and curvature error) and reports more than 2 dB PSNR gain and PSNR above 26 dB at 90% confidence. Triage systems, by contrast, are judged by an asymmetric pair of quantities at a fixed operating point. AIRIS-TB reports 80% workload reduction at 1.57% false-negative rate, against a radiologist baseline of 1.85%. ARDA reports 97.0% sensitivity and a 0% clinically important miss rate across 45 rural sites.

Our claim is that these two goals come apart. A lens that maximizes average PSNR under tolerance can still let a minority of units erase the faint, low-contrast cues that separate "no finding" from mild disease. Those minority units dominate the fleet's worst-case false-negative rate.

Concretely, let the optical surface parameters be $\theta$, the digital classifier weights $w$, the shared threshold $\tau$, and a unit's manufacturing error vector $\delta \sim P$, where $P$ is the factory tolerance distribution. Let $\mathrm{FNR}_\delta(\tau)$ and $\mathrm{WLR}_\delta(\tau)$ be that unit's false-negative rate and workload reduction (fraction of all exams auto-cleared). We solve

$$\max_{\theta, w, \tau} \; \mathbb{E}_{\delta \sim P}\big[\mathrm{WLR}_\delta(\tau)\big] \quad \text{subject to} \quad Q_{0.95}\big(\mathrm{FNR}_\delta(\tau)\big) \le \alpha,$$

where $Q_{0.95}$ is the 95th percentile over units and $\alpha$ is the clinical ceiling (we use 1%, close to the human baseline in AIRIS-TB).

In training, both rates are replaced by sigmoid-smoothed counts so they can be differentiated. The percentile constraint is replaced by its conditional value-at-risk, the mean false-negative rate over the worst 5% of sampled units, which is a standard tractable stand-in for a quantile. The constraint enters as a Lagrangian penalty whose weight grows during training.

What is new is the target: a fleet-wide guaranteed operating point with no per-unit calibration, applied to the physical lens itself. Neither the optics literature, which targets fidelity, nor the triage literature, which assumes a single clean sensor, has optimized for this.

Research Program

The application regime has to be one where cheap optics genuinely sit between the patient and the model. We start with a simple, data-accessible case: a fixed-focus, low-cost camera module that digitizes radiograph films or display screens in clinics that lack direct digital transfer. This is not hypothetical. The ankylosing spondylitis tool was evaluated on smartphone photos of physical radiographs and reached AUC 0.904, against 0.96 on the multicenter digital images. The capture path costs real accuracy even with a good phone camera.

The underlying images come from CheXpert: 224,316 radiographs, plus a 500-study test set labeled by consensus of five radiologists. The triage task is "No Finding" versus any abnormality, matching the clear-the-normals workflow. Secondary per-finding tasks are pleural effusion and edema.

The optical model is a two- or three-element molded-plastic lens simulated with the ray-tracing approach of Tolerance-Aware Deep Optics, which matches Zemax to within 1 µm, under that paper's four error types. We widen the tolerance ranges to what injection molding plausibly delivers rather than precision glass. Fundus imaging is the natural second regime, because there the cheap lens forms the image of the eye directly. We treat it as a follow-on, since the only fundus data in our file, the Aravind post-deployment set, is not public.

The method has three stages. First, nominal pre-training, in the two-stage spirit of Dai et al. Second, joint fine-tuning of lens surfaces and classifier under the fleet-yield objective. Each step samples a batch of simulated units, renders each training image through each unit, and computes the worst-tail false-negative rate. Third, a factory-fixed threshold chosen once on a held-out set of simulated units.

We borrow one idea from BOSON-1: adaptively oversampling the "corner case" variations that most hurt performance. For us, these are the error configurations producing the worst false-negative rates. Uniform Monte Carlo would rarely visit the worst 5% tail that the objective cares about.

We also add a self-check that needs no patient labels. Each unit photographs a printed resolution target at manufacture, and a small model predicts from that capture whether the unit falls inside the region of error space where the yield guarantee holds. The motivation is two observations. ARDA relies on "ungradable" escalation as a safety valve. The gestational-age system drops frames whose predicted uncertainty exceeds a threshold. It is unknown whether such software uncertainty can tell hardware blur apart from genuine clinical ambiguity. A factory-side physical check sidesteps that question for out-of-tolerance units, and we separately measure whether the classifier's own uncertainty would have caught them.

Baselines

The naive reference point designs the lens for nominal sharpness, trains the classifier on nominal renders, and sets the threshold on the nominal unit. This is what a team following SACoD would build. SACoD co-designs a phase mask with a network but leaves tolerance out of the loop, and it documents a 4% simulation-to-hardware accuracy gap. We expect this baseline to show a wide spread of per-unit false-negative rates. It is beaten if our worst-tail false-negative rate is lower at equal mean workload reduction.

The upper reference is a precision lens with negligible tolerance spread, in front of a DenseNet121 trained on CheXpert. Irvin et al. report AUC 0.97 for pleural effusion and 0.94 for edema with this backbone. For the "No Finding" triage operating point, no reported result was found on Lacuna, so we will measure it ourselves. This reference tells us how much of the precision-optics performance the cheap design recovers. It is not something we expect to beat.

The strongest existing method is Tolerance-Aware Deep Optics run as published: tolerance-aware lens plus reconstruction network, trained with its spot loss and PSF-similarity loss, followed by a DenseNet121 classifier trained on its reconstructions under the same sampled tolerances. Its reported numbers are more than 2 dB PSNR over baselines and PSNR above 26 dB at 90% confidence. These are fidelity metrics, and no triage result was found on Lacuna. It is the strongest competitor because it already optimizes the physical lens under exactly our error model, so any gain we show must come from the triage-specific target rather than from tolerance awareness as such.

The most dangerous baseline is backend-only physical augmentation. The lens is the nominal cheap design, frozen. The classifier is trained on images rendered through sampled manufacturing errors from the same ray tracer, using the same fleet-yield objective. This isolates our key ingredient, which is shaping the optics rather than only the software.

The second ablation reverses this. It keeps the optics optimization but swaps the worst-tail constraint for a mean false-negative-rate constraint or a symmetric cross-entropy loss. This isolates the yield objective.

Finally, as an oracle that our method should approach without needing, we run per-unit calibration in the style of NTKD. NTKD fine-tunes the digital backend on 10% of the data for each fabricated unit and restores CIFAR-10 accuracy from 47.3% to 74.9%. In our run, each unit's threshold and final layer are re-fit on labeled scans.

All arms share the same CheXpert splits, the same backbone, the same number of sampled units per step, and the same hyperparameter search budget. Every arm is evaluated on the same held-out set of 1,000 simulated units, drawn once with a fixed seed.

Why not just…?

Why not just train the classifier on physically simulated blur and keep an ordinary cheap lens? This is the real threat, and the factorial design is built to settle it. Generic Gaussian blur is clearly insufficient. Dai et al. show that real tolerance errors produce spatially varying, asymmetric PSFs, and NTKD's uncompensated hardware dropped to 47.3% from a 75.6% simulated baseline. Augmentation with ray-traced PSFs is a much stronger version, however. The argument against it is information loss: if some units null out the frequency band carrying faint pleural or interstitial cues, no backend can restore it. That is the premise behind Dai's PSF-similarity loss and behind the file's observation that post-hoc calibration cannot recover detail lost at the sensor. If backend-only augmentation matches our worst-tail false-negative rate within confidence intervals, we will report that the optics contribution is null and recommend the cheaper path.

Why not just set the shared threshold conservatively enough that the worst unit is safe? This always achieves safety, but at a price in workload reduction, and that price is the point of the study. Our metric, mean cleared fraction at a guaranteed 95th-percentile false-negative rate, reports exactly how much clearing this policy forfeits. We include it as a threshold-only variant of the naive baseline. If its workload loss turns out small, the problem is easier than we think, and that finding is useful in itself.

Why not just buy precision optics? The specialist gap is concentrated where budgets are tightest: AIRIS-TB cites one radiologist per ten million people in some regions. SACoD argues that precision refractive assemblies cannot meet the cost and size needed for portable devices. We nonetheless keep the precision lens as an explicit upper reference, so the cost-performance tradeoff is visible rather than assumed.

Why not just calibrate each unit after manufacture? NTKD shows this works but needs labeled data for every unit. Across deployments the size of ARDA's 45 sites, that means curated clinical labels per device, which rural programs cannot produce. We run it as the oracle. Success means coming within about one percentage point of its workload reduction at the same false-negative ceiling, without its labels.

Proposed Experiments and Decisive Checks

The main experiment is the factorial of three optics designs (nominal, tolerance-aware for PSNR, tolerance-aware for fleet yield) by two classifier objectives (mean, worst-tail). Each cell reports the 95th-percentile per-unit false-negative rate, mean workload reduction at $\alpha = 1\%$, and the full distribution of per-unit AUC, all on CheXpert's radiologist-consensus test set rendered through 1,000 held-out simulated units. Confidence intervals come from bootstrapping over both patients and units.

The decisive comparison is fleet-yield optics plus worst-tail classifier against nominal optics plus worst-tail classifier. A gap of at least five points in workload reduction at equal guaranteed false-negative rate would justify optical co-design. No gap would falsify it.

The second check is whether PSNR and triage yield actually diverge. We correlate per-unit PSNR from Dai's design with per-unit false-negative rate. If the correlation is near-perfect, the fidelity objective already suffices and our contribution narrows to the thresholding policy.

Third, a small physical validation following Dai et al.'s adjustable-perturbation lens setup. We dial in about 20 error configurations spanning the tolerance distribution, including the adaptively found worst cases, and photograph printed and displayed CheXpert test radiographs. This checks that simulated per-unit false-negative rates predict measured ones. SACoD's 4% gap is the size of discrepancy we must rule out.

Fourth, we test the self-check. On the physical captures and on deliberately out-of-tolerance simulated units, we measure whether the resolution-target predictor and the classifier's own uncertainty each flag units that violate the false-negative ceiling. This answers whether software uncertainty can stand in for a factory test or is fooled by hardware blur.