A powder X-ray diffraction pattern is a one-dimensional trace of scattered intensity against angle. It is the main way chemists check what crystal they have made when the material only exists as a powder. Because the measurement collapses three-dimensional crystal information onto a single axis, reflections from different crystal planes often land on top of each other. This is called peak overlap. When a sample also contains a small amount of an unexpected second phase, such as an unreacted precursor or a side product, its peaks frequently fall under the main phase's peaks and disappear into them. The newest machine-learning structure solvers, which turn a pattern into a crystal structure, assume the whole pattern comes from one pure crystal. Faced with a contaminated pattern, they bend the predicted structure until it absorbs the impurity intensity. They report a good fit and a wrong answer.
We propose to treat impurity detection as reconstruction-based anomaly detection, with one important change. In the usual approach, a model trained only on "normal" data rebuilds the input, and whatever it fails to rebuild is flagged. Here the rebuild comes from a structure solver followed by the exact physics simulator for single-phase diffraction, not from a learned decoder. The simulator can only produce patterns that a single crystal could produce, so it cannot reproduce a second phase. The remaining failure, where the solver distorts the structure to soak up impurity intensity, is handled by a fitting objective that uses the fact that a mixture's intensities add and are never negative. The claim is that this pipeline finds impurity peaks that sit underneath main-phase peaks, where a standard refinement difference curve, pattern-space autoencoders, and peak-search heuristics all go blind.
Core Claim
Reconstruction-based anomaly detectors fail through the "identical shortcut": a decoder with enough capacity reproduces the anomaly, so the residual goes to zero. DRÆM attacks this in images by adding a discriminator trained on synthetic defects. Powder diffraction allows a stronger fix. The forward map from structure to pattern is known exactly. If it is used as the decoder, the shortcut cannot happen at the decoding step. It can only happen upstream, when structure solving and refinement move atoms and profile parameters to absorb foreign intensity. That is the same failure deCIFer documents as deceptively low $R_{wp}$ (the weighted profile residual crystallographers use to judge fit) for physically wrong structures.
We block that remaining path with an asymmetric refinement loss. Let $y_i$ be the observed intensity at angle $i$, $f_i(\theta)$ the simulated single-phase intensity for structure and profile parameters $\theta$, $\sigma_i$ the counting uncertainty, and $r_i = y_i - f_i(\theta)$. We minimize
$$ \mathcal{L}(\theta) = \sum_i \Big[ \tfrac{r_i^2}{\sigma_i^2}\,\mathbb{1}[r_i<0] \;+\; \lambda \tfrac{|r_i|}{\sigma_i}\,\mathbb{1}[r_i\ge 0] \Big]. $$
The two terms do different jobs. The first term penalizes overshoot quadratically. A single-phase model should never predict more intensity than was observed, because extra phases only add intensity. The second term penalizes undershoot with an L1 term of weight $\lambda$, which tolerates sparse, large positive residuals. That lets an impurity peak stay unexplained instead of being absorbed. The L1 term is also what stops the trivial solution of shrinking the scale factor to zero.
The anomaly map is the positive part of the residual. It is computed across $K$ candidate structures sampled from the generator and refined under $\mathcal{L}$. We take the median over candidates whose fit is acceptable. This means a localized flag requires agreement across alternative structures that explain the data equally well, which is how we account for homometry (different structures giving the same pattern).
Relation to Prior Work
The generative structure solvers supply the method. PXRDGen combines a contrastive pattern encoder, a flow-matching generator over atomic positions, and automated Rietveld refinement. It reports 82% one-sample and 96% twenty-sample match rates on MP-20. deCIFer generates crystallographic text files autoregressively from patterns. XtalNet uses contrastive pretraining plus diffusion for metal-organic frameworks. CrystalNet regresses continuous electron density. All four assume a single pure phase and none flags what it cannot explain. We reuse a PXRDGen-style generator as the candidate source and change what happens after sampling.
deCIFer also motivates the work. Its observation that $R_{wp}$ stays low for wrong structures is the crystallographic form of the identical shortcut, and it is the failure our loss targets.
The anomaly-detection literature supplies the framing and constrains the design. The diffusion anomaly localization survey describes the noise paradox: too little noise keeps the anomaly, and too much erases the normal content. Peak broadening from the instrument is exactly the kind of "noise" a pattern-space diffusion model would confuse with impurities. That argues against learned decoders in pattern space and for the physics decoder. DRÆM shows that a discriminator trained on synthetic anomalies breaks the shortcut in images. We keep its synthetic-anomaly idea for calibration only, not as the detector. SplatPose uses an explicit 3D representation with pose alignment before subtracting residuals. Our design is analogous: an explicit crystal plus a physical simulator stands in for a learned decoder.
The composition autoencoder for stability screening is the only materials anomaly detector in the file. It deliberately ignores diffraction, so it addresses a different question.
On the physics, Lightfoot et al. and Tedesco et al. explain why overlap makes separating individual intensities ill-posed. This is why peak-by-peak methods cannot find buried impurities.
Research Program
The work has three parts.
A controlled mixture benchmark. We take MP-20 test structures as main phases and add one impurity phase at weight fractions from 0.5% to 20%. Impurities are drawn from held-out chemical families. Patterns are simulated with realistic profile broadening, background, and counting noise. Each mixture gets an overlap index: the fraction of impurity integrated intensity lying within one full width at half maximum of any main-phase reflection. Per-angle ground-truth masks mark where impurity intensity exceeds the noise level. This lets every method be scored separately on exposed and buried impurity peaks.
The detector. We pair a flow-matching structure generator trained only on pure MP-20 patterns with a differentiable powder simulator. Refinement minimizes $\mathcal{L}$, with $\lambda$ and the overlap tolerance calibrated on DRÆM-style "phantom phases": random lattices with random occupancies, never real compounds. If PXRDGen weights are available we use them directly. Otherwise we retrain the generator, and before any anomaly experiment we report its pure-phase MP-20 match rate against the published 82%/96%.
Analysis of when detection is possible at all. Some impurity intensity added under a main reflection can be reproduced exactly by a different single-phase structure in the same cell. No single-pattern method can detect that. We estimate this ceiling by asking whether an oracle single-phase refinement, started from the true structure but free to move, can absorb the added intensity. We report detection relative to this ceiling as well as in absolute terms.
Baselines
Naive reference. We flag a pattern when the single-phase best-fit $R_{wp}$ exceeds a threshold, and flag individual peaks that cannot be indexed to the solved cell. This is what a practitioner does without new tools. It should work for exposed impurity peaks and fail under overlap. Any method that does not beat it on buried peaks has nothing to offer.
Strongest existing pipeline. We run PXRDGen-style solving, take the best of $K$ candidates, apply standard least-squares Rietveld refinement, and read the difference curve as the anomaly map. It is the strongest competitor because it uses the same generator, the same simulator and the same $K$; it differs from our method only in the fitting objective and the aggregation across candidates. Both pipelines get identical compute, identical candidate counts, and the same tuning budget for thresholds on a shared validation split.
No reported result for impurity detection or localization in powder diffraction was found on Lacuna. The only published reference numbers are PXRDGen's pure-phase match rates and DRÆM's image-domain localization AP of 68.4 on MVTec AD, which is an analogy, not a comparable target. Beating this baseline means a statistically significant gain in per-angle localization AP on buried peaks (overlap index above 0.5) at equal performance on exposed peaks, with main-phase match rate on contaminated patterns no worse than the baseline's.
Learned pattern-space reconstruction. A 1D diffusion model and a 1D autoencoder are trained on pure patterns, with residuals optionally fed to a DRÆM-style discriminator trained on phantom phases. They get the same training data and a comparable parameter budget. This baseline tests whether the physics decoder matters.
Supervised classifier. A segmentation network is trained on simulated mixtures. It is evaluated on held-out impurity families and on real patterns.
Ablations. We remove the asymmetric loss (symmetric least squares with $K$ candidates), set $K=1$, and give the method the oracle main-phase structure. The oracle run separates errors from structure solving from errors in residual extraction.
Why not just…?
Why not just inspect the Rietveld difference curve? When the main phase is known and correctly refined, this is close to what we do, and the oracle ablation will show how close. Its weakness is that least squares is free to shift atomic positions, occupancies, and peak-shape parameters to absorb buried impurity intensity. deCIFer's low-$R_{wp}$ observation is indirect evidence that this happens. The comparison stratified by overlap index settles the question directly. If least-squares and asymmetric refinement perform the same on buried peaks, the core claim is false and we will say so.
Why not just fit peaks and flag the ones that cannot be indexed? Peaks that cannot be indexed are by definition the exposed ones. Buried impurity peaks share positions with main-phase reflections, and under overlap the individual intensities cannot be recovered stably (Tedesco et al.). The naive baseline implements this approach, and we expect it to collapse as the overlap index rises.
Why not just train a supervised classifier on simulated mixtures? This is a real threat for impurity types seen in training, and it may win there. Its risk is the open set of possible impurities plus the gap between simulated and real data, which opXRD documents for real measurements. The held-out-family and real-data tests decide the matter. If the classifier matches us on held-out families, our contribution shrinks to interpretability and to the ability to recover impurity fractions.
Why not just use an autoencoder or diffusion model on the pattern itself? A learned decoder can reproduce a weak impurity peak, which is the identical shortcut. The noise paradox described in the diffusion survey also predicts that such a model will confuse broadening with anomalies. The learned pattern-space baseline tests both predictions.
Proposed Experiments
The decisive check is localization AP plotted against overlap index, at fixed impurity fraction, for our method, the least-squares difference curve, and the oracle ceiling. The core claim predicts three things. All methods should agree at low overlap. The baseline should degrade sharply as overlap rises. Our method should stay closer to the ceiling.
Secondary measurements are:
- pattern-level detection AUROC;
- error in recovered impurity weight fraction;
- main-phase match rate on contaminated patterns;
- a demonstration that standard refinement reaches low $R_{wp}$ on mixtures while getting the structure wrong.
For real data, we use the 912 fully labeled opXRD patterns for false-positive calibration, since they are presumed mostly single-phase. A crystallographer will then annotate a subset blind, so that flagged regions can be checked against expert judgment. Because ground-truth impurity labels for real patterns do not exist in the file, these real-data results will be reported as agreement with experts, not as accuracy.
References
- DRÆM - A discriminatively trained reconstruction embedding for surface anomaly detection
- Tackling Real-World Crystal Structure Prediction from Powder X-ray Diffraction Data
- Powder diffraction crystal structure determination using generative models
- End‐to‐End Crystal Structure Prediction from Powder X‐Ray Diffraction
- Towards end-to-end structure determination from x-ray diffraction data using deep learning
- Denoising Diffusion Models for Anomaly Localization in Medical Images
- SplatPose & Detect: Pose-Agnostic 3D Anomaly Detection
- Anomaly-Detection-Driven Screening of Thermodynamic Stability from Composition Descriptors Alone
- Determination of molecular crystal structures on a laboratory powder diffractometer
- New Aspects of Direct Space Structure Solution from Powder Diffraction Data
- opXRD: Open Experimental Powder X‐Ray Diffraction Database