Cryo-electron microscopy (cryo-EM) images frozen protein molecules with an electron beam. A structure is recovered by averaging tens of thousands of very noisy particle images. Increasingly, the aim is not one average but the whole range of shapes a molecule adopts. Deep generative models such as cryoDRGN learn a small "latent" code for each particle image that is meant to describe its shape. The beam, however, also shakes and deforms the ice the particles sit in. Standard tools correct this motion only approximately, and whatever they miss blurs the particle image. The shape model has no way to tell that blur apart from a real change in shape, so it can quietly encode imaging artifacts as biology.
Wearable and radar vital-sign monitoring has already solved a version of this problem. A tiny, stable signal (a heartbeat) is buried under a large, time-varying disturbance (body motion). The most effective fix there is a consistency constraint: whatever is real must stay the same across time windows, and whatever changes is treated as artifact. We carry that principle into cryo-EM. A particle's conformation is effectively fixed during a one-to-two-second exposure. Its motion blur and radiation damage are not: they are largest in the first frames and change steadily with dose. We propose to encode separate time blocks of each particle's exposure movie and force their shape codes to agree. A small per-block "nuisance" code is left free to absorb blur and damage.
Core Claim
Recent physiological sensing work removes motion without clean labels by exploiting what stays constant across windows. Contrast-Phys enforces that the pulse frequency stays the same across facial regions and across 10–30 second windows. MoRe-Fi splits the latent space of a variational encoder–decoder into motion and respiration parts. Our claim is that the cryo-EM exposure movie contains the same exploitable structure. Cryo-EM heterogeneity models currently throw it away by collapsing each movie into a single motion-corrected, dose-weighted image.
CryoDRGN encodes each static particle image into a continuous latent and decodes it with a coordinate network. It relies entirely on upstream correction of beam-induced motion, and the research file flags the risk that residual blur is absorbed into the latent. Our model, which we call block-consistent cryoDRGN, treats the movie differently. Contiguous dose blocks of the same particle are separate views of one conformation. They are rendered through one shared conformation code and several block-specific nuisance codes.
The prediction is directional and falsifiable. Relative to cryoDRGN on the best available polished stacks, the conformation latent should become less predictable from measured motion and dose nuisances. It should keep, or improve, recovery of known conformational states.
Research Program
Each extracted particle $i$ is kept as $B$ contiguous frame blocks rather than one summed image, with $B$ between 3 and 5 (for example early, middle and late dose). This follows motion correction by the strongest existing tool. A shared encoder maps each block image $y_{ib}$ to a conformation code $z_{ib}$ and a low-dimensional nuisance code $n_{ib}$.
The decoder renders a projection of the volume $V(\bar z_i)$ at the particle's known pose, using the block-averaged conformation code $\bar z_i$. It then passes that projection through the block's contrast transfer function (the microscope's known optical filter). Finally it applies a small parametric degradation $D(n_{ib})$: an anisotropic blur kernel plus residual shift, and a frequency envelope initialized from the dose-dependent relative B-factors of Bayesian polishing. The training objective is
$$ \mathcal{L}_i=\sum_{b=1}^{B}\big\| y_{ib}-D(n_{ib})\,C_{ib}\,P_{\phi_i}V(\bar z_i)\big\|^2+\lambda\sum_{b=1}^{B}\|z_{ib}-\bar z_i\|^2+\beta\,\mathrm{KL}. $$
Here $C_{ib}$ is the block's transfer function and $P_{\phi_i}$ the projection at pose $\phi_i$. $\lambda$ weights cross-block agreement, and the KL term is the usual variational regularizer on both codes.
The nuisance code is deliberately weak: a handful of parameters feeding a fixed physical degradation model. It therefore cannot carry conformational information. Two things prevent the trivial solution of a constant, uninformative $\bar z$. First, every block must still be reconstructed from the shared volume. Second, $\lambda$ is chosen to be the largest value at which held-out reconstruction error does not rise.
Contiguous blocks are essential, and this is the point where we depart from Cryo-CARE. Cryo-CARE splits movies into even and odd frames to get two independent noisy copies for self-supervised denoising. Both copies then span the whole trajectory and share nearly the same blur. A consistency constraint on even/odd pairs removes noise but cannot remove motion. Contiguous early and late blocks differ precisely in motion and damage, which is what we want the constraint to reject. We borrow Cryo-CARE's idea of treating frame subsets as independent observations, but for a different purpose.
Baselines
The naive reference point is cryoDRGN trained on particles from whole-micrograph patch motion correction (MotionCor2) with standard dose weighting. This is common practice, and Scheres' particle polishing showed it leaves substantial local motion uncorrected for sub-megadalton particles.
The strongest existing competitor is cryoDRGN on stacks processed with RELION-3 Bayesian polishing. Its Gaussian-process trajectories and Fourier-cylinder B-factors are the most thorough motion and damage correction in the file. On β-galactosidase (EMPIAR-10061) it reports a resolution improvement of 0.25 Å over prior tools and the same resolution with 30% fewer particles. On the same dataset, Scheres' earlier polishing reports an improvement from 4.3 Å to 4.0 Å, and on γ-secretase (EMPIAR-10194) from 4.9 Å to 4.5 Å. No reported cryoDRGN latent-quality number was found on Lacuna for EMPIAR-10076. The only heterogeneity number on that dataset is the 87.9% cluster agreement that Generative Priors for Cryo-EM Image Reconstruction reports for 2×-compressed reconstructions. We use it only as a scale reference, not as a like-for-like target.
For fairness, our method and every cryoDRGN baseline start from the same Bayesian-polished trajectories, poses, particle set, latent dimension, decoder architecture and GPU-hour budget. Each receives the same hyperparameter sweep of roughly ten configurations. Beating this baseline means two things. The conformation latent's predictability of nuisance variables must fall by a margin outside seed-to-seed variation. At the same time, recovery of known states and the resolution of latent-selected subsets must be at least as good.
Two further baselines isolate the key ingredient. The first is post hoc nuisance removal: cryoDRGN conditioned on, or with latents regressed against, per-particle measured nuisances. These nuisances are polishing trajectory length, early-frame displacement, position within the grid hole, defocus and a local ice-thickness proxy.
The second is the adversarial approach of Choi et al., which trains a discriminator to predict those nuisances from the latent and penalizes the encoder when it succeeds. We also run three ablations. Setting $\lambda=0$ keeps the block architecture but removes agreement. An even/odd split replaces contiguous blocks. Removing the nuisance code tests whether a degradation pathway is needed at all.
Why not just…?
Why not just rely on Bayesian polishing and assume the residual blur is negligible? This is the most serious threat, and the first experiment is designed to settle it before any modeling claim is made. Bayesian polishing still models only in-plane translation, as the file notes. It does not capture out-of-plane tilting from ice doming or early non-linear bursts. So some residual blur plausibly remains, especially for small particles like γ-secretase. But "plausibly" is not evidence. If, after Bayesian polishing, a held-out regressor cannot predict nuisances from cryoDRGN latents better than chance on the homogeneous β-galactosidase control, the problem is already solved upstream. In that case we would report a negative result rather than proceed.
Why not just regress the nuisance variables out of the latent afterwards, or condition the decoder on them? This is cheap and should be tried, which is why it is a baseline. Its weaknesses are twofold. It only removes nuisances we can measure, and out-of-plane tilt is exactly the one we cannot. It also removes biology that happens to correlate with a nuisance: particles near the grid-hole edge or in thicker ice may genuinely favor certain states. The block-consistency constraint needs no nuisance labels and never penalizes that kind of correlation. On synthetic data where a true state is deliberately correlated with position in the hole, regression should erase that state while our method keeps it. That experiment decides the question.
Why not just use an adversarial nuisance discriminator, as radar respiration monitoring does? The same confound applies, more sharply. An adversary forces the latent to be independent of the nuisance even when biology and nuisance are truly correlated. Choi et al. could accept this because locomotion and chest expansion are causally separate, an assumption that does not hold in cryo-EM ice. We include the adversary as a baseline and test it in the same correlated synthetic setting.
Why not just sort blurred particles into junk classes with multi-class 3D classification? CryoDRGN's authors show that discrete classes fragment continuous landscapes such as the assembling 50S ribosome. Polishing papers also show that "bad" particles still carry recoverable signal. Discarding them both loses data and fails to separate blur from shape within the particles that remain.
Proposed Experiments
The first experiment is diagnostic. On EMPIAR-10061 (a rigid tetramer, so the latent should hold almost no real conformational variance) and EMPIAR-10194 (a 170 kDa membrane complex where per-frame signal is weakest), we train cryoDRGN on MotionCor2 stacks and on Bayesian-polished stacks. We then measure the held-out R² of predicting each nuisance variable from the latent. A substantial R² on β-galactosidase is direct evidence that artifacts leak into the latent.
The second experiment is synthetic. We simulate movies from a two-state or continuous-hinge volume and inject beam-induced motion with three components: smooth translation, an early non-linear burst, and per-particle out-of-plane tilt that grows with dose. In some conditions we correlate the true state with position in the hole. Because ground truth is known, we can measure how well the learned latent correlates with the true conformation coordinate versus the injected motion, and compare all methods and ablations.
The third experiment is on real heterogeneous data: EMPIAR-10076, if raw movies are deposited, otherwise an equivalent deposited movie dataset of a flexible complex. The measures are recovery of the published assembly states, reported as cluster agreement with the reference cryoDRGN classes; cross-block latent agreement on held-out particles; and gold-standard half-map resolution of volumes reconstructed from latent-selected subsets.
The decisive check has two parts. First, the method must lower nuisance predictability on β-galactosidase and in the synthetic sets, relative to the Bayesian-polished baseline. Second, it must preserve the correlated-with-position state in the synthetic test that regression and adversarial baselines erase. If the $\lambda=0$ ablation matches the full model, or contiguous blocks do no better than even/odd blocks, then the gain is not coming from temporal consistency and the central claim fails.
References
- Contrast-Phys: Unsupervised Video-Based Remote Physiological Measurement via Spatiotemporal Contrast
- MoRe-Fi: Motion-robust and Fine-grained Respiration Monitoring via Deep-Learning UWB Radar
- Uncovering structural ensembles from single-particle cryo-EM data using cryoDRGN
- A Bayesian approach to beam-induced motion correction in cryo-EM single-particle analysis
- Cryo-CARE: Content-Aware Image Restoration for Cryo-Transmission Electron Microscopy Data
- Beam-induced motion correction for sub-megadalton cryo-EM particles
- Generative Priors for Cryo-EM Image Reconstruction
- Remote Respiration Monitoring of Moving Person Using Radio Signals