Catching Optimizers That Game Optical Scorers in Antireflection Design

When engineers design antireflection surfaces, such as the tapered "moth-eye" cones etched into infrared lenses, they rarely optimize against the real physics. Solving Maxwell's equations accurately for every candidate shape is too slow. Instead they optimize against a cheap stand-in scorer. That stand-in might be effective medium theory, which treats a layer of tiny cones as a smooth blend of air and crystal. It might be a coarse-grid simulation, or a neural network trained to predict transmission. Language-model evaluation has already shown what happens when you push hard against a stand-in scorer: the optimizer finds shortcuts. Factuality metrics rise by 0.1 to 0.2 points when a content-free sentence is appended (Ramprasad et al.), and one dialogue metric rewards copying the previous speaker turn 76% of the time (Nohejl et al.). We argue that optical inverse design has the same problem and has not measured it. An optimizer searching over cone shapes will drift toward geometries where its scorer is wrong, for example periods near the onset of diffraction, needle-thin tips, or features aligned with the simulation grid. The scorer then reports near-perfect transmission that the physical device cannot deliver.

This note proposes two things. The first is to measure this gaming directly, as a function of how hard the optimizer pushes. The second is to test one specific remedy borrowed from language-model evaluation: split the single transmission score into separate physical terms, attach to each term an explicit condition for when that term can be trusted, and combine the terms outside the scorer with a rule that sends any design outside the trusted region to the expensive solver. The remedy is tested against simpler fixes under an equal budget, and the note states in advance what result would mean the simpler fixes are enough.

Core Claim

Surrogate-driven design of graded-index antireflection structures (structures whose refractive index changes gradually from air to the substrate) is subject to Goodhart's law in a measurable way. The gap between the score the proxy reports and the true full-wave transmission grows with optimization pressure. The gap concentrates in identifiable regimes where the physics breaks down, and it is invisible when surrogates are validated only on typical, randomly sampled shapes.

The second part of the claim is conditional. A decomposed scorer that knows where each of its parts is valid should shrink that gap at a fixed budget of high-fidelity solves, and it should also report which physical assumption a failed design violated. The proposed test is that it does both better than an uncertainty penalty or than post-hoc checking of the top designs.

We define the gaming gap for an optimizer run as

$$G(k) = \mathbb{E}\big[\,\hat{T}(x_k) - T(x_k)\,\big],$$

where $x_k$ is the best design after $k$ optimizer steps, $\hat{T}$ is the band-averaged transmittance the proxy predicts, and $T$ is the value from a fine-mesh full-wave solver. The quantity of interest is how $G$ grows with $k$, not its value at any single point.

Where This Sits

Three prior lines of work bear on this, each in a different role.

One line supplies the physical testbed. Teslenko et al. etched moth-eye microstructures into zinc selenide (ZnSe, a common infrared lens material) using femtosecond laser ablation followed by wet etching. Transmittance rose from about 83.5% to an average of 93.2% over 6–18 μm, peaking at 98% at 16 μm. They reached this through roughly 400 trial samples guided by forward simulation, with no automated inverse design, so the question of scorer gaming never arose. Jiang et al. and the TiO₂ nanocone work ( Bio-inspired TiO2 nano-cone antireflection layer for the optical performance improvement of VO2 thermochromic smart windows) establish that tapered cones hold low reflection across wide bands and angles. Lü et al. show the stepped, discrete version that fabrication often forces, and those steps reintroduce small reflections at each interface.

A second line supplies the strongest existing optimizer. Multi-BOWS warm-starts fine-mesh (2 nm) simulations from a Pareto frontier found with coarse-mesh (40 nm) simulations, and it found that double-sided nanocones win for transparent electromagnetic shielding. It treats the coarse solver as a trustworthy but cheap approximation. It does not ask whether the optimizer exploits coarse-mesh artifacts, and it notes that gains shrink as the parameter space grows. We build directly on it as the baseline to beat.

A third line motivates the remedy and also constrains it. DnA-Eval splits judging into separate aspects and moves aggregation to an external calculator, raising GPT-4's agreement with humans on FairEval from 42.4% to 59.1%. HD-Eval uses a hierarchy of criteria with a trained, transparent aggregator. ExPerT shows that decomposed atomic scoring resists a confidence trick that inflates a holistic judge's scores by 12.9%.

The constraining lesson comes from Nohejl et al.. The evaluator that correlates best with humans on natural data is not the most robust. Accuracy on benign inputs therefore tells you nothing about robustness under optimization, which is why we test scorers on optimizer-produced designs rather than on held-out random shapes.

The closest optical-adversarial work, Chen et al., attacks a multi-layer diffractive optical network used as a classifier, cutting MNIST accuracy from 98% to 44%. That work perturbs the inputs to fixed hardware. We study the design loop, where the optimizer itself is the adversary of its scorer.

Research Program

The testbed is a broadband antireflection surface for ZnSe over 6–18 μm. It is modeled as a two-dimensional periodic array of tapered structures. Each structure is parameterized by its period, its height, and a free profile giving fill fraction versus depth, represented with 8–32 control points. An optional mode requires the profile to be realized as N discrete fabrication steps. Ground truth comes from a converged fine-mesh full-wave solver, with convergence checked by mesh refinement. We also run a planar thin-film control. For flat stacks the transfer-matrix method, which propagates light layer by layer with exact matrices, has no approximation error, so no gaming is expected there. That control tells us whether any gap we measure is an artifact of our measurement pipeline.

We study three proxy scorers, each with a different characteristic way of being wrong. The first is effective medium theory feeding a transfer-matrix calculation. It is fast, and it ignores diffraction, near-field coupling at sharp tips, and the breakdown of the smooth-blend assumption as the period approaches λ/n. The second is coarse-mesh full-wave simulation, as used in Multi-BOWS, which is vulnerable to mesh-aligned features and numerical dispersion. The third is a neural surrogate trained on randomly sampled smooth profiles, which is vulnerable to extrapolation. Each scorer is optimized with gradient ascent where gradients exist, with an evolution strategy (CMA-ES), and with Bayesian optimization, which chooses each new design using a statistical model of the scores seen so far. We record $G(k)$ along every run.

Two further artifacts come out of this stage. One is a stress suite, the optical analogue of the dialogue-attack suite: hand-constructed families of pathological geometries (periods just below and just above the diffraction onset, tips below 50 nm, spiky index profiles, grid-aligned steps) with fine-mesh ground truth for each. The other is a failure taxonomy that labels each gamed design by the assumption it violated.

The remedy is a decomposed scorer. Its terms are:

  • zero-order transmission of the graded profile, computed with effective medium theory and transfer matrices;
  • an estimate of scattering and diffraction loss, trusted only when the period is well below λ/n across the band;
  • step reflection from discretizing the profile into N layers;
  • robustness to fabrication error, estimated by averaging the score over perturbed sidewall angles and etch depths;
  • a manufacturability term covering aspect ratio and minimum feature size.

Each term reports its value together with a flag saying whether it can be trusted for the current design. Aggregation happens outside the scorer. Wavelength bands where every term is trusted are scored by the decomposed model. Any band or design that fails a trust flag is either scored conservatively (assigned transmission no higher than a flat reference surface) or placed in a queue for the fine solver. This routing is the key ingredient: the decomposition decides where expensive computation is spent, rather than spreading it uniformly.

Baselines

The naive reference point is to optimize each proxy with no safeguards and then verify only the final design with the fine solver. Its measured gap $G$ defines how large the problem is. A second fixed reference is the measured ZnSe result from Teslenko et al.: 93.2% average transmittance over 6–18 μm and 98% peak, against 83.5% untreated. Those are experimental numbers, so we compare them only as a plausibility anchor against our simulated fine-solver values, never as a direct score.

The strongest existing method is Multi-BOWS (Multi-BOWS). Its authors report 3.2%–89.9% improvement in normalized area under the Pareto frontier over single-fidelity Bayesian optimization, and 0.5%–10.3% over prior multi-fidelity methods. We reimplement it with coarse-then-fine warm starts, on the same parameterization and the same two objectives: band-averaged transmittance and robustness to fabrication error. The comparison is fair because every method gets the same budget of fine-solver calls, the same wall-clock limit for cheap calls, and the same hyperparameter tuning budget. Beating Multi-BOWS means a higher fine-solver-verified area under the Pareto frontier, and a smaller $G$, at equal fine-solver budget, across at least five random seeds.

A third baseline is an ensemble-uncertainty penalty: several neural surrogates are trained, and the score is lowered wherever they disagree. This is the standard safeguard in model-based optimization. A fourth is hard constraints alone: period below λ_min/n and a cap on aspect ratio, with no decomposition.

Two ablations isolate what the decomposition contributes. The first sums the decomposed terms into one scalar with no trust flags, which tests whether splitting the score matters by itself. The second keeps a single global trust check and routing rule without the per-term decomposition, which tests whether per-mechanism attribution matters beyond the routing.

Why not just…?

Why not just check the top designs with the fine solver afterwards? This is the most serious threat. If only a few top designs are gamed, post-hoc checking of the top k is cheap and nearly sufficient. The case for our approach rests on the empirical finding we expect: under strong optimization pressure, most top-ranked designs come from exploited regimes, so post-hoc checking spends its budget confirming failures and has nothing better left to fall back on. We measure the gamed fraction of the top-k directly. If it stays below roughly 20%, we will report that post-hoc checking is adequate for this problem class.

Why not just penalize ensemble disagreement? Ensembles catch random extrapolation error, but not bias that every member shares. Effective medium theory is deterministic and wrong in the same direction near the diffraction onset. Coarse meshes err consistently on grid-aligned features. The language-model literature points the same way: a well-aligned evaluator can still be gamed systematically (Nohejl et al.). The comparison against the ensemble baseline on the stress suite settles this. If the ensemble matches our gap reduction on the neural-surrogate arm, we concede that arm and keep the claim only for the physics-based proxies.

Why not just impose the known constraints, such as period below λ/n? This is also a real threat, since much of effective medium theory's failure region is textbook knowledge. We expect hard constraints to remove diffraction gaming but to leave tip-coupling, discretization, and mesh gaming untouched, and to over-restrict designs near the constraint boundary where good solutions may sit. If constraints alone close the gap, then the decomposed scorer's remaining value is only diagnostic, and the paper will say so. The failure taxonomy then becomes the contribution.

Why not just use exact transfer-matrix optimization? For flat stacks this is correct, and it is exactly our planar control. Graded cone structures are not flat stacks, and the literature favors them for bandwidth, angular tolerance, and damage resistance (Teslenko et al.). So the approximation, and with it the opportunity for gaming, cannot be avoided in the regime that matters.

Proposed Experiments and Decisive Checks

The first experiment establishes that the phenomenon exists. On the ZnSe task, we plot $G(k)$ for each combination of proxy and optimizer. The claim holds if $G$ rises with optimization pressure for all three 2D proxies while staying near zero for the planar control, and if at least half of the gamed designs fall into a small number of taxonomy categories. No reported result on optical scorer gaming rates was found on Lacuna, so these would be first measurements.

The second experiment is the stress-suite analogue of attack success rate. It reports the fraction of pathological geometries each proxy scores above the flat-surface reference while the fine solver scores them below it. It also reports how that fraction relates to each proxy's accuracy on random shapes, as a direct test of whether the alignment-robustness paradox appears in optics.

The third experiment compares remedies at equal fine-solver budget against Multi-BOWS, the ensemble penalty, hard constraints, and the two ablations. It reports verified area under the Pareto frontier, $G$, and the number of fine-solver calls wasted on gamed designs.

The fourth experiment tests the method under a shift of task. We transfer the method unchanged to the two-objective transparent-shielding problem from Multi-BOWS, where the three-layer and nanocone structures define the search space.

The decisive result is this: the decomposed, trust-gated scorer lowers $G$ and raises verified Pareto area relative to Multi-BOWS and to hard constraints in both tasks, and the flags-removed ablation loses most of that advantage. If instead constraints alone or top-k checking match it, the contribution narrows to the measurement study and the taxonomy. That is still a first quantification of Goodhart effects in nanophotonic design, but it would be reported as such and not dressed up as a new method.