Motion Graphs of Flexible Molecules from Randomized X-ray Laser Simulation

X-ray free-electron lasers (XFELs) can image a single virus or protein assembly at room temperature by firing a pulse so short, a few to about a hundred femtoseconds, that the particle scatters light before the pulse blows it apart. This regime is called "diffraction before destruction." Each particle yields exactly one noisy 2D diffraction pattern, taken at an unknown 3D orientation, and then it is gone. Current reconstruction software, including the fastest neural pipeline, assumes that every particle is the same rigid object. Biology rarely cooperates with that assumption. Carboxysomes vary in size and shape, viruses flex, and assemblies coexist in several states. The room-temperature snapshots that make XFELs valuable are therefore processed as though the motion they record were noise.

We propose to train the network that reads diffraction patterns entirely in a heavily randomized simulator, so that it learns to separate "which way the particle is turned" from "what shape the particle is in," and to organize the inferred shapes into a motion graph. A motion graph here is a network of representative states in which an edge is kept only if the step between two decoded structures is small and actually populated by observed particles. The simulator randomizes beam, sample and detector nuisances in the way robotics randomizes friction and lighting. The motion graph borrows its structure from character animation and manipulation planning, where discrete nodes and edges that enforce feasibility replace free movement through a continuous space. The output is a map of the pathways a molecule takes between states, recovered from unsynchronized single shots of samples that no light pulse can trigger.

Core Claim

The central difficulty is identifiability, not throughput. When particles differ in shape, a network trained without labels on real patterns can explain a shape change as a rotation, or a rotation as a shape change, and nothing in the reconstruction loss stops it. Simulation supplies the missing supervision: exact orientations, true electron densities and true phases for every synthetic shot. Our claim is that an encoder pretrained this way transfers to real XFEL data without any labeled real shots and without collapsing pose into shape. The condition is that the randomization spans the real beam, detector and sample nuisances, and also spans a wide family of synthetic shapes and deformations. Once pose is pinned down, a physics-based decoder trained on the real data can learn a conformational coordinate. A motion graph built over that coordinate recovers true transition orderings more faithfully than tracing paths through the raw continuous latent space. The graph claim is the one most likely to fail, and we treat it as a hypothesis tested against simple alternatives rather than as a design commitment.

Research Program

The work has three parts, in order of importance.

Randomized simulator. We start from the source-to-detector chain of simS2E, described in A comprehensive simulation framework for imaging single particles and biomolecules at the European X-ray Free-Electron Laser. That work used simulation only in the forward direction, to study how pulse length and optics affect resolution. Its findings set our randomization ranges. Mirror aperturing cuts pulse energy by up to a factor of 2.3. Compton scattering washes out high-resolution speckle. Pulse duration controls how much damage occurs during the exposure: 9 fs pulses reached about 7.2 Å, while 30 fs pulses degraded.

The expensive atomistic damage module cannot run for millions of shots, so we fit cheap surrogate parameterizations to it: a resolution-dependent intensity loss and an incoherent background. We then randomize the following quantities per shot:

  • pulse fluence, photon energy and spot profile;
  • particle offset from the beam center;
  • Compton background;
  • per-pixel detector gain and dark offsets;
  • missing central data;
  • solvent or water shells;
  • multi-particle clusters and empty frames.

Fluence ranges are anchored to the values reported for the real LCLS runs, which used 1.2–1.7 keV photons and 2–4 mJ pulses in a spot of roughly 1.3 by 1.7 μm, per Diffraction data from aerosolized Coliphage PR772 virus particles imaged with the Linac Coherent Light Source. Shapes are randomized as well. We sample polyhedra of varying size and faceting, elastic and hinge-like deformations of generic scaffolds, and atomic ensembles from molecular dynamics.

Encoder training with privileged information. We use a privileged training scheme. A training-only branch sees simulator ground truth and supervises a student encoder that sees only the noisy, phaseless pattern. This adapts Asymmetric Actor Critic for Image-Based Robot Learning, where a critic with access to true simulator state helped an image-only policy transfer zero-shot to a real arm. The privileged branch is given the true orientation, true density and true phase.

The student encoder outputs three things: an orientation on the space of 3D rotations, a conformational code, and a shot-type label (single hit, cluster, or empty). The orientation loss is supervised directly. The code is trained to predict held-out views of the same simulated conformation, so it cannot carry orientation information.

Real-data decoder and motion graph. On real data we freeze or lightly tune the orientation head. We then fit an implicit density field conditioned on the conformational code, using the differentiable scattering decoder of Scalable 3D reconstruction for X-ray single particle imaging with online machine learning (X-RAI), which streams 161 images per second but assumes a homogeneous sample.

Motion-graph nodes are medoids of the per-shot codes. An edge between two nodes survives only if two conditions hold. First, the decoded densities differ by a small transport cost. Second, enough real shots fall along the latent segment between the nodes to show the transition is populated. Pathways are minimum-cost routes through this graph. The edge rule is the analogue of the "seamless transition" test in video motion graphs such as Audio-driven Neural Gesture Reenactment with Video Motion Graphs. It also echoes how Regrasp Maps for Sequential Manipulation Planning merges regions and connects them only through feasible transitions.

How this relates to prior work. The closest method is CryoDRGN, which learns a continuous conformational latent for cryo-EM. It differs from our setting in three ways: it uses linear projection images rather than phaseless diffraction, it needs poses from a prior homogeneous reconstruction, and it traces paths through the latent space heuristically. We do not assume known poses, and we replace free latent interpolation with a data-supported graph.

CryoBench motivates the graph. It found that continuous latents often look plausible when decoded but are tangled when checked with neighborhood-based fidelity metrics, especially for disordered motion such as its IgG-RL dataset.

HEA-Map builds discrete conformation maps from computed energy scores. It shows that discrete maps help exploration, but it never touches experimental data.

Baselines

Naive reference. The simplest option is homogeneous X-RAI run exactly as published. Every shot is treated as one rigid structure, with no simulation pretraining. It uses the same shots, the same GPU and the same wall-clock budget as our method. It is the right floor because it is the fastest working pipeline on real SPI data, and it shows what ignoring heterogeneity costs. On real data the file reports X-RAI at 18.95 nm resolution for PR772 and 4.56 nm for gold nanoparticles, and sub-2 nm on synthetic ATP synthase, where M-TIP and Dragonfly timed out or failed to converge (X-RAI). To beat it on a heterogeneous sample, our per-conformation reconstructions must be sharper than its single average. On a homogeneous sample, our method must not be worse.

Strongest competitor. The strongest competitor is X-RAI extended with a per-shot conformational code: a CryoDRGN-style latent added to X-RAI's self-supervised encoder and decoder, trained only on the target data. This is the real threat. It needs no simulator, and it may already disentangle pose and shape well enough. It gets the same architecture capacity, the same real data and the same number of gradient steps as our real-data stage. On synthetic benchmarks it also gets a hyperparameter search matched to ours.

The most comparable published heterogeneity number is RECOVAR's Adjusted Rand Index of 0.968 on CryoBench Ribosembly (CryoBench). That result is on cryo-EM projections with given poses, not XFEL diffraction, so it is an upper reference and not a direct target. We found no reported heterogeneity result for XFEL single-particle imaging on Lacuna.

Beating this competitor means lower orientation error and better latent-fidelity scores on the XFEL-simulated CryoBench ensembles, under a realistic-nuisance test set that neither method trained on.

Ablations of the two key ingredients. The first ablation replaces wide randomization with narrow, nominal-parameter simulation. It isolates whether the diversity of the simulator, rather than simulation itself, drives transfer. The second ablation keeps our latent but replaces the motion graph in two ways: with a plain k-nearest-neighbor graph, and with straight-line latent interpolation. It isolates whether the feasibility-and-support edge rule adds anything.

Why not just…?

Why not just train on the real data with a physics decoder, as X-RAI does, and add a latent code? This may work, and it is our strongest baseline for that reason. Our argument against it is identifiability. Without ground-truth poses, a shape change and a rotation can explain the same pattern equally well. CryoDRGN avoids this only by importing poses from a separate homogeneous reconstruction, and no such reconstruction exists for flexible XFEL samples. The simulated benchmarks settle the question directly: we measure each method's orientation error against ground truth. If the self-supervised variant matches ours on those numbers, the simulator is unnecessary for pose, and we would reframe the contribution around shot-type classification and the graph.

Why not just build a high-fidelity simulator instead of randomizing? simS2E itself shows that shot-to-shot pulse variation and optics losses of up to a factor of 2.3 are properties of the real facility. Its atomistic modules are also far too slow for million-shot training sets. In robotics, (CAD)$^2$RL transferred by widening variety rather than chasing realism. Randomization carries a known risk as well: overly wide ranges can make the model overly cautious. The narrow-versus-wide ablation, plus a fidelity-matched nominal simulator given the same compute, will show where the optimum lies.

Why not just cluster the latent or connect nearest neighbors, instead of building a motion graph? For purely compositional mixtures such as Ribosembly, clustering may be all that is needed, and we will report that honestly. The graph earns its place only on continuous or disordered motion. There, nearest-neighbor edges can bridge unpopulated gaps and produce the tangled paths CryoBench documents. The path-ordering test on molecular-dynamics trajectories decides this.

Why not just run multi-class EMC? Expansion-Maximization-Compression (Veit Elser) searches exhaustively over orientations, and the cost of that search multiplies with each added state. X-RAI reports that M-TIP and Dragonfly failed on large targets. We will still run multi-class EMC on the smallest synthetic benchmark as an accuracy reference.

Proposed Experiments

Synthetic benchmark. We convert CryoBench's IgG-1D, IgG-RL, Spike-MD and Ribosembly atomic ensembles into XFEL diffraction patterns. The test set uses a held-out nuisance distribution fitted to the real LCLS parameters and disjoint from the training randomization. Metrics are:

  • geodesic orientation error;
  • per-image Fourier shell correlation;
  • CryoBench's Neighborhood Similarity and Information Imbalance for latent fidelity;
  • Adjusted Rand Index on Ribosembly.

For Spike-MD we add a graph-specific metric: the rank correlation between the node ordering along recovered shortest paths and the ground-truth molecular-dynamics time ordering.

Real-data transfer. Here there is no pose ground truth, so we rely on external anchors. For carboxysomes (CXIDB ID 25, about 70,000 patterns), the recovered size coordinate should reproduce the independently measured 115 ± 26 nm diameter distribution. Per-state projections should match or exceed the reported 18.2 nm (A data set from flash X-ray imaging of carboxysomes).

For PR772 (285,944 classified hits), the published hit counts disagree: PCA identified about 8,000 hits and Diffusion Maps over 37,000 (Coherent soft X-ray diffraction imaging of coliphage PR772 at the Linac coherent light source). Our shot-type head will be scored against the later manually curated labels. Reconstruction resolution will be compared against X-RAI's 18.95 nm.

Decisive checks. Three results decide the project. First, if the self-supervised latent baseline matches our orientation error on held-out-nuisance synthetic data, the domain-randomization claim fails. Second, if the motion graph does not beat nearest-neighbor and linear-interpolation paths on Spike-MD ordering and IgG-RL fidelity, the graph claim fails. Third, if the carboxysome size coordinate does not track the independently measured distribution, zero-shot transfer to real beamline data has not been shown, whatever the synthetic results say.