Lot fingerprints in SEM strength prediction, treated like ancestry confounding

Models that predict a material's strength from scanning electron microscope images can look far more accurate than they are. A scanning electron microscope (SEM) builds images at sub-micron resolution by sweeping a focused electron beam over a surface. Datasets for this task usually hold tens of thousands of images but only a few dozen physical batches ("lots"), and each lot carries one measured strength. A network can therefore score well by recognizing which lot an image came from: its preparation, coating, contrast and imaging session. It never has to learn what makes material strong.

Genetic risk prediction has the same structure. A polygenic score trained on European cohorts partly encodes ancestry background, and it degrades on other populations. That field has built tools for the problem: evaluation that holds out whole populations, and models that split a representation into an ancestry part and a disease part. This note proposes transferring both tools to SEM-based property prediction, with one change forced by the physics. Because strength is constant within a lot, lot identity cannot simply be removed from the representation, since that would also remove the signal. The method instead requires that lots with the same strength be indistinguishable in the representation used for prediction.

Core Claim

Our first claim is about evaluation. Image-level train/test splits on SEM strength data mostly measure lot memorization. The honest analogue of cross-ancestry testing is leave-lot-out evaluation, with error reported per lot and for the worst lot. We will also run a permutation test that shuffles strengths across lots, which directly measures how much "accuracy" comes from recognizing lots.

Our second claim is about method. A representation trained to predict strength while carrying no lot information among lots of similar strength should transfer to unseen lots better than three alternatives:

  • an unconstrained convolutional network;
  • a network whose lot information is removed outright, which destroys signal;
  • a pooled, per-lot regression on handcrafted texture features.

The last of these is the real threat, and the experiments are built around beating it.

Grounding in prior work

The application regime comes from Predicting compressive strength of consolidated molecular solids using computer vision and deep learning. That paper pairs 69,894 SEM micrographs of TATB, an explosive molecular solid, with peak compressive stress across 30 manufacturing lots. It compares DenseNet121 against Random Forests built on BSIF and KAZE texture descriptors. It motivates this proposal in two ways. First, its regime finding: Random Forest beats deep learning with up to 24% lower RMSE when few lots are available, which is what you would expect if the deep model is fitting lot-specific detail. Second, its observation that fine-scale texture carries the predictive signal. The file does not say whether its reported errors come from lot-held-out splits. That ambiguity is exactly what our first experiment resolves.

The geotechnical papers Micro-Mechanism of Strength for Cement-Treated Soil Based on the SEM Experiment and Strength Model for Cement-Stabilized Marine Clay supply interpretable descriptors computed from binarized SEM images:

  • image porosity;
  • shape factor (how round pores or particles are);
  • particle grading entropy (how spread out the particle sizes are);
  • directional entropy (how ordered their orientations are).

These papers also supply the cautionary result that motivates the whole note. The fitted grading-entropy optimum moves from 0.3125 in Hangzhou soil to 0.5 in Taizhou soil, and the marine-clay paper had to build a new composite parameter. Per-source curve fitting does not transfer.

The method comes from Improving genetic risk prediction across diverse population by disentangling ancestry representations (DisPred). DisPred splits genotype embeddings into an ancestry vector and a disease vector using supervised contrastive losses, then predicts from the disease vector alone. It reports higher AUC than PRS-Lasso in African and admixed cohorts. We reuse its two-part encoder and contrastive training. We differ in the constraint we impose. DisPred makes the disease vector unconditionally ancestry-blind. That is safe when ancestry and disease are only partly linked, but here lot fully determines the label.

Two theory papers constrain the design. External Validity: From Do-Calculus to Transportability Across Populations gives the formal condition under which a relationship learned in one population carries to another, but it assumes a known causal graph that we do not have. Invariant Ancestry Search recovers causal ancestors from a union of minimally invariant predictor sets across environments, but it warns that results depend heavily on the power of the independence tests. With roughly 30 lots, a full search for invariant sets over image features would be badly underpowered. For that reason we use an invariance penalty during training rather than an invariance search, and we reserve IAS-style testing for the handful of interpretable descriptors.

Research Program

We train an encoder that produces two vectors per image. The lot vector $z_a$ is trained with a supervised contrastive loss to cluster by lot, which gives the model a designated place for lot fingerprints. The property vector $z_d$ feeds a regression head for lot strength $y$.

The key ingredient is a conditional penalty. Partition lots into strength bins $b$, chosen so that each bin holds several lots with similar measured stress. The training objective is

$$\mathcal{L} = \mathcal{L}_{\text{reg}}(y, f(z_d)) + \alpha\,\mathcal{L}_{\text{con}}(z_a, \text{lot}) - \lambda\,\mathcal{L}_{\text{adv}}\big(\text{lot} \mid z_d, b\big).$$

Here $\mathcal{L}_{\text{reg}}$ is squared error on log stress and $\mathcal{L}_{\text{con}}$ is the contrastive lot loss on $z_a$. $\mathcal{L}_{\text{adv}}$ is the cross-entropy of an adversary that tries to tell lots apart from $z_d$ within a bin $b$, and its sign is flipped through a gradient-reversal layer. The weights $\alpha$ and $\lambda$ are chosen by nested leave-lot-out validation.

The intent is simple. Two lots of equal strength should look the same to the predictor, so anything that separates them in $z_d$ is a fingerprint. Lots in different bins may still differ in $z_d$, so the strength signal survives. Predictions are pooled per lot (median over images), because the label exists only at the lot level.

For interpretability, we compute the four descriptors from binarized TATB micrographs (porosity, shape factor, grading entropy, directional entropy) and test whether $z_d$ is linearly predictable from them. We also test whether $z_a$ is better explained by acquisition statistics such as mean intensity and contrast. On these low-dimensional descriptors, and on the BET surface area and particle size measurements available for 17 lots, we run IAS using lot groups (by manufacturing process, or by bulk-measurement tercile) as environments. The goal is to see which descriptors keep a stable relationship to strength.

Baselines

The naive reference is the non-image mean predictor. On the TATB data it is reported at RMSE 419 and MAPE 26% in Predicting compressive strength…. Under leave-lot-out evaluation, its leave-one-out analogue (predict the mean of the training lots) is the floor that any image model must clear.

The strongest reported image model is DenseNet121, at RMSE 206 and MAPE 10% in the same paper. The Random Forest on BSIF/KAZE features is reported at RMSE 271 and MAPE 13%. The file gives no split protocol for these numbers, and no reported leave-lot-out result was found on Lacuna for this dataset. We will therefore retrain both models ourselves under identical leave-one-lot-out folds, using the same images, the same pretrained initialization for the network, and an equal hyperparameter search budget. Model selection uses nested folds only.

The baseline most likely to win is a deliberately simple one. Pool BSIF texture features (and, separately, frozen DenseNet121 embeddings) to one mean vector per lot, then fit ridge regression on about 29 lots. This model cannot memorize image-level fingerprints at all, so it is the fair test of whether our constraint adds anything beyond "average over the lot."

Two ablations isolate the key ingredient. In the first, $\lambda = 0$: a two-headed encoder without the conditional penalty. In the second, the penalty is unconditional, with the adversary removing all lot information from $z_d$ as in a direct transfer of DisPred. We predict that the second ablation collapses toward the mean predictor.

Beating the field means two things under leave-lot-out evaluation:

  • lower lot-level RMSE than both the lot-pooled ridge and the retrained DenseNet;
  • lower worst-lot absolute percentage error, across repeated seeds.

Bootstrapping over lots gives the uncertainty.

Why not just…?

Why not just average features per lot and fit a small regression? This is the most serious threat, and it may match our method. Pooling removes image-level memorization. It does not remove lot-level fingerprints, such as a coating difference that happens to correlate with strength across 30 lots. It also discards within-lot variation that could teach the encoder which texture changes matter. If lot-pooled ridge equals our model on leave-lot-out RMSE, we will report that the conditional penalty is unnecessary on this dataset. The evaluation protocol would then stand as the main contribution.

Why not just remove lot identity adversarially, as standard domain generalization does? When labels are constant within a source, a representation that cannot identify the source cannot predict the label. The unconditional ablation tests this directly. DisPred's setting avoids the problem only because ancestry and phenotype are partly independent, which is not the case here.

Why not just recalibrate per lot, the way ethnicity-specific diagnostic thresholds fixed MODY testing? That work shifted BMI cut-offs per ethnic group, which doubled detection from 8.8% to 16%. It works because group labels and outcome data exist for each group. Recalibrating to a new lot requires destructive strength tests on that lot, which is what image prediction is meant to avoid. We will nonetheless report a one-measurement-per-lot intercept correction as an upper reference, to show how much of the leave-lot-out error is a simple offset.

Why not just fit handcrafted descriptor curves, as in the cement-soil work? The Hangzhou/Taizhou shift in the optimal grading entropy shows that such curves are source-specific. We include descriptor-only ridge as a baseline and test its invariance across lot groups with IAS.

Proposed Experiments

The first check decides whether the premise holds. We train DenseNet121 under image-level random splits and under leave-lot-out splits, and we run a placebo in which strengths are permuted across lots. If the image-split model still "predicts" permuted strengths well, and leave-lot-out error is much higher than image-split error, lot fingerprinting is confirmed. If the two errors are close, the confounding story is weak and the method is not needed.

The second check is the main comparison. All baselines and ablations are compared in leave-one-lot-out and leave-extreme-lots-out settings. The extreme setting holds out the strongest and weakest lots, which mirrors deploying to an unrepresented population. We report per-lot RMSE, MAPE and worst-lot error.

The third check is a leakage probe. A linear probe on $z_d$ should fail to identify lots within a strength bin, while the same probe on $z_a$ should succeed. The probe on $z_d$ should also be predictable from the porosity, entropy and shape descriptors.

A secondary transfer test on cement-treated soil would use the PCAS-style measurements from the Xu et al. papers, with leave-one-soil-out evaluation. It depends on obtaining the raw images. Since only three soils are involved, it would be treated as a qualitative stress test rather than a benchmark.