Torsion-Space Outlier Synthesis for Macromolecular Generators

Generative models that build three-dimensional macromolecules — proteins, long polymers, ligand ensembles — have escaped the cost explosion of manipulating every atomic coordinate by moving to torsion space: instead of diffusing all $3N$ Cartesian coordinates, they freeze stiff bond lengths and angles and generate only the rotatable dihedral angles, cutting the dimensionality of a drug-like molecule from 132 numbers to about 8 and letting a protein side-chain packer run with 3 million instead of 208 million parameters Torsional Diffusion for Molecular Conformer Generation, DiffPack: A Torsional Diffusion Model for Autoregressive Protein Side-Chain Packing. But these generators are trained only on physically valid structures, so nothing tells them where physical plausibility ends — the same open-space overconfidence problem documented for image classifiers in Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In vision, the fix is to manufacture synthetic outlier samples near the boundary of the training data and penalize confident predictions on them; the best current methods do this either by perturbing features along directions where real data does not vary Geometrically Constrained Outlier Synthesis or by decoding imagined near-outlier images with a diffusion model, at a cost of roughly 10 GPU-hours per 100,000 samples Dream the Impossible: Outlier Imagination with Diffusion Models.

The proposal in two sentences: because torsion space is low-dimensional and compact, we can synthesize physically auditable near-boundary negatives — torsion configurations that look statistically plausible but decode into structures with steric clashes, broken chirality, or unphysical rotamers — with a single-step consistency generator at a tiny fraction of DREAM-OOD's cost, and use them as negative supervision for macromolecular torsional generators. And because the static benchmarks these generators are scored on are themselves flawed and memorizable, we evaluate the effect on a procedurally generated benchmark whose instances cannot exist in any training corpus. Macromolecular scalability is the core mechanism here; synthetic outlier generation is the training signal; high-resolution multi-scale scaling sets the architecture the signal is injected into; and benchmark contamination is the evaluation constraint that makes the measurement trustworthy.

Core Claim

A single-step (one function evaluation) consistency generator, trained in torsion space and constrained to sample only within a thin statistical band of near-boundary but physically invalid configurations, can serve as negative supervision that contracts a macromolecular generator's production of unphysical conformations — without degrading its accuracy on valid structures and without the iterative-diffusion cost that makes pixel-space outlier imagination expensive. Crucially, this effect is only credibly measurable on dynamically generated evaluation instances, because the field's static splits are compromised twice over: by evaluation bugs (GEOM-Drugs Revisited: Toward More Chemically Accurate Benchmarks for 3D Molecule Generation found valency-table errors, fractured reference geometries, and force-field evaluators too coarse to separate leading models) and by memorization that survives the probes usually used to detect it (Test of Time: Rethinking Temporal Signal of Benchmark Contamination showed rephrasing alone drops training-set influence traceability from 77.5% to 17.5% and erases post-cutoff decay entirely).

No existing work does this. The closest attempts each stop one step short: Geometrically Constrained Outlier Synthesis confines its conformal shell to linear PCA directions in image-classifier feature space, which would cut through the nonlinear folds of a torsional hypertorus; GOCM: Single-Step Graph Outlier Synthesis via Origin Consistency Model achieves the single-step synthesis we want but only over one-dimensional graph-embedding vectors, with no notion of 3D coordinates, symmetry, or stereochemistry; DREAM-OOD makes its negatives visually auditable but only in 2D pixels; and ConfHit: Conformal Generative Design with Oracle-Free Guarantees prunes bad small-molecule candidates after generation with density-ratio conformal testing rather than shaping the generator during training, and never touches full macromolecules.

Research Program

The first component is the negative generator itself. We train a small origin-consistency model in the torsional latent space of a frozen positive generator (DiffPack-style, on a GearNet-Edge invariant backbone encoder), following the 1-NFE recipe that let GOCM cut graph-outlier generation from 172 to 71 seconds. Raw consistency samples are uninformative, so we filter them through a conformal shell on the torus: let $s(\tau)$ be a learned nonconformity score (an SE(3)-invariant energy-like head trained on valid torsion configurations), and keep only samples $\tilde{\tau}$ satisfying

$$q_{0.95} \;\le\; s(\tilde{\tau}) \;\le\; q_{0.99},$$

where $q_{0.95}$ and $q_{0.99}$ are the 95th and 99th percentiles of $s$ on a held-out calibration set of valid structures. This mirrors the shell construction of GCOS but replaces its linear PCA directions with the learned score's own level sets, which is what lets the band follow the manifold's nonlinear curvature. Unlike in vision, every retained negative is then physically auditable: we decode $\tilde{\tau}$ to all-atom coordinates through the fixed bond template and label it with deterministic physics checks — atom-clash counts, chirality and valency verification, rotamer plausibility — so each synthetic negative comes with a machine-checkable reason it is invalid. This directly addresses GCOS's acknowledged weakness that linear subspaces can synthesize nonsense negatives, because nonsense is filtered out by the physics labeler rather than trusted.

The second component injects these negatives into a multi-scale generator. Following DiffPack's autoregressive $\chi_1 \to \chi_2 \to \chi_3 \to \chi_4$ schedule and Hierarchical Multi-Scale Molecular Conformer Generation's scaffold-first upsampling, we apply the negative loss per scale, targeting the compounding "whip effect" where small errors in the backbone-adjacent angle cascade into large terminal-atom displacements. This also resolves, empirically, the open tension between single-step consistency synthesis and multi-scale cascades noted across GOCM and Make a Cheap Scaling: A Self-Cascade Diffusion Model for Higher-Resolution Adaptation: our hypothesis is that one-step consistency suffices for negatives (they only need to be near-boundary and labeled) while the positive pathway still needs the cascade, and we test whether 1-step and 20-step negative generation differ at matched negative counts.

The third component is the evaluation apparatus. Following the template of DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks — which generates DAG-structured problems on the fly so they cannot be scraped, and on which static-benchmark leaders like phi-1.5 scored near zero — we build a procedural side-chain and conformer benchmark: novel backbones assembled combinatorially from fragment libraries with difficulty tiers scaling segment count and linker flexibility, each instance's ground truth checked on the fly by the same physics labeler used for negatives. Instances are generated per evaluation run and never published. We complement this with a memorization audit in the spirit of Test of Time: we measure whether models can reproduce near-duplicate training conformers (RMSD to nearest training neighbor) and whether template perturbations — the structural analog of paraphrasing — launder that memorization out of view. P$^4$Bench shows cryptographic verification works only for NP-style algebra; our physics labeler plays the verifier role for continuous geometry, which that line explicitly leaves open.

Baselines

The "do nothing" reference point is an unregularized DiffPack-style torsional generator: the same autoregressive architecture, backbone encoder, and training data as ours, with the negative-loss weight set to zero. This isolates what negative supervision adds, since DiffPack reports roughly a 12–13% improvement in side-chain torsional angle accuracy over AttnPacker on CASP13 and CASP14 with only 3 million parameters (versus AttnPacker's 208 million) and zero bond-length violations — the quality bar our regularized model must not degrade. We retrain this baseline ourselves under matched data, steps, and tuning budget rather than quoting its numbers for our setting.

The strongest existing competitor is post-hoc conformal pruning in the manner of ConfHit, which used density ratios and nested conformal testing to prune batches of 15 generated molecules down to 2–5 certified hits (a 30–50% reduction) while keeping empirical failure below the user-set $\alpha$, explicitly avoiding the empty-set collapse of Bonferroni corrections. We implement its filtering stage over our generator's raw outputs with the same physics labeler, giving it identical information; it is the strongest competitor because it requires no retraining and already carries distribution-free guarantees. Beating it means achieving a lower rate of physically invalid retained samples at equal or better valid-sample yield per unit compute — i.e., showing that shaping the sampler beats filtering the sampler.

The key-ingredient ablation replaces our conformal-shell, physics-labeled negatives with cheap negatives at matched count and matched training compute: isotropic Gaussian perturbations of torsion features in the VOS style, and randomly corrupted decoded structures (clash injection, angle scrambling). If the cheap variants match the shell-based negatives, the proposal's central ingredient is unnecessary and the idea should be cut down to "corruption augmentation." We will also run a 20-step iterative-diffusion negative generator at matched wall-clock to test whether the 1-step consistency design matters.

Why not just…?

Why not just filter outputs with force fields afterward? This is the most serious alternative, and we run it as the ConfHit-style baseline rather than arguing around it. The file's evidence says pure physics filtering falls short at the frontier: GEOM-Drugs Revisited shows diffusion models now exceed MMFF94's structural precision, so a classical force field can misgrade exactly the near-boundary structures that matter; CREST-style sampling needs days of supercomputer time per GEOM, energy-annotated molecular conformations for property prediction and molecular generation; and FlexiFlow found alternative low-energy poses that CREST and RDKit missed. Filtering also never improves the sampler, so invalid samples still burn compute. Still, if filtering matches our retained-sample purity at equal total compute, the training-time mechanism is not worth its complexity — and the head-to-head experiment is designed to answer exactly that.

Why not just add Gaussian noise or random corruptions as negatives? Torsion space is low-dimensional, so unlike high-dimensional image features, naive perturbation might already land usefully near the boundary — this is a genuine threat, not a strawman, which is why it is the primary ablation rather than a dismissed footnote. What the existing evidence suggests is that unconstrained sampling mostly produces trivially far or intruding negatives: on Stanford Dogs, VOS's Gaussian outliers gave a 5.00% FPR95 where GCOS's constrained shell gave 0.00%, with GCOS averaging 93.47% AUROC across four near-OOD datasets Geometrically Constrained Outlier Synthesis. The physically-invalid-but-statistically-plausible band is thin, and we expect random corruption to be separable without teaching the model anything about the boundary. The matched-budget ablation settles it either way.

Why not just use temporal splits to handle contamination? Test of Time directly rules this out as sufficient: paraphrasing removed temporal decay on LiveCodeBench, models scored +2.19 points higher on post-cutoff physics under multi-step rephrasing, and influence traceability collapsed from 77.5% to 17.5%. The structural analog — perturbing templates and backbones — would launder memorized coordinates the same way, which is why our audit includes a template-perturbation arm and why the procedural benchmark, whose instances are generated fresh per run in the DyVal spirit, is the primary evaluation rather than a supplement.

Proposed Experiments and Decisive Checks

Experiment one targets side-chain packing on CASP13 and CASP14 under DiffPack's protocol, scoring angle accuracy, clash counts, and bond violations, with the unregularized and ConfHit-filtered variants as comparators; success is unregularized-level angle accuracy with a materially lower clash and invalid-rotamer rate, and the dynamic procedural benchmark as the honest version of the same comparison. Experiment two targets conformer ensembles on GEOM-Drugs under the corrected evaluation protocol of GEOM-Drugs Revisited (kekulized representations, GFN2-xTB-based geometric metrics), where the strongest reported configuration in the file is MSGEN wrapping GeoDiff at 90.41% recall coverage and 0.8424 Å matching error Hierarchical Multi-Scale Molecular Conformer Generation — no reported result found on Lacuna for any outlier-exposure regularizer on this benchmark, which is precisely the open slot. Experiment three is the detection study: can a validity head, trained with our negatives, separate near-boundary invalid structures from valid ones (AUROC), benchmarked against GCOS-in-feature-space, VOS-Gaussian, and random-corruption training on identical data. Experiment four is the contamination audit: train-set conformer retrieval, template-perturbation laundering, and the static-versus-dynamic score gap, testing whether any model that looks strong on static splits weakens on freshly generated instances as DyVal exposed for language models. The decisive check for the whole proposal is a single plot: physical-invalidity rate versus valid-sample quality across all baselines at matched compute — if the shell-based negatives do not dominate the cheap-negative ablation on that plot, the mechanism does not earn its complexity, and the fallback contribution is the dynamic macromolecular benchmark plus the memorization audit, which the field lacks entirely.