ConfRover: Simultaneous Modeling of Protein Conformation and Dynamics via Autoregression

Yuning Shen, Lihao Wang, Huizhuo Yuan, Yan Wang, Bangji Yang, Quanquan Gu

TL;DR

CONFROVER is the first deep generative framework that simultaneously learns both protein conformation distributions and temporal dynamics from molecular dynamics (MD) trajectories. By framing conformation generation as an autoregressive sequence task, it unifies trajectory simulation, time-independent ensemble sampling, and conformation interpolation into a single model, achieving up to a 52× speedup over classical MD simulations.

What Problem Does This Paper Address?

Proteins are highly flexible molecules whose biological functions depend on their ability to shift between different 3D shapes, known as conformations. While Molecular Dynamics (MD) simulations are the gold standard for capturing these movements, they are computationally grueling and often get stuck in local energy minima.

To accelerate this, researchers have turned to deep generative models. However, existing models suffer from a split personality: they either generate static, time-independent "ensembles" of shapes (completely ignoring the chronological path/kinetics) or they generate trajectories using rigid, non-autoregressive architectures that cannot handle variable-length sequences or perform direct, unconditional sampling.

The authors introduce CONFROVER to bridge this gap, creating a unified framework capable of simulating continuous physical motion, sampling independent equilibrium states, and interpolating transition pathways between known structures.

Methods

CONFROVER’s core innovation is treating MD trajectories as a continuous sequence of frames and modeling them autoregressively:

$$p(\mathbf{x}^{1:L}|\mathcal{P}) = \prod_{l=1}^{L} p(\mathbf{x}^l|\mathbf{x}^{<l},\mathcal{P})$$

Instead of discretizing 3D coordinates into tokens (which introduces quantization errors), CONFROVER couples a causal sequence model with a continuous structure decoder.

Figure 1: Key ideas of CONFROVER. (A) Conformation generation tasks with various conditioning configurations. Each block denotes a frame and arrows indicates the sequential dependencies among frames from autoregressive formulation. Initial conditioning frames are outlined in black. In conformation interpolation, the last frame is repositioned and prepended to the first frame for proper sequential dependencies. (B) CONFROVER models each frame as a conditional distribution given preceding frames. Sequential dependencies are captured through latent variables h, and conformations are sampled from a diffusion decoder, conditioned on the updated latent.

The architecture consists of three modular layers:

  1. Encoding Layer: Uses a frozen OPENFOLD model to extract sequence-specific embeddings. Concurrently, a FrameEncoder processes the 3D coordinates of each frame (via $C_\beta$ pairwise distances) into a frame-level latent representation ($\mathbf{h}^l$).
  2. Trajectory Module: Models spatial and temporal dependencies across frames. It interleaves StructuralUpdate blocks (using Pairformer layers for spatial reasoning) with TemporalUpdate blocks (leveraging a lightweight, Llama-style causal transformer to compute attention across the time dimension).
  3. Structure Decoder: An $SE(3)$ diffusion model based on CONFDIFF that iteratively denoises random noise into precise 3D backbone structures, conditioned on the updated temporal latents. A lightweight AngleResNet then predicts the torsional angles to reconstruct full-atom geometry.

To support multiple tasks, the authors use a hybrid training strategy. For time-independent sampling, they mask input frames and apply identity attention to disable inter-frame communication. For interpolation, they simply reorder the sequence, prepending the target end-frame to the start-frame.

Figure 3: Architecture overview. (A) Encoding Layer embeds protein sequence and input structure to each frame as a frame latent representation hl, comprised of single and pair embeddings; (B) The Trajectory Module then updates frame latent hl using interleaved structural and temporal update blocks; (C) A diffusion-based Structure Decoder learns to denoise noisy conformations conditioned on the updated frame latent hl; during inference, it samples conformations from the prior distribution. See Appendix D.1 for details.

Datasets

  • ATLAS: A large-scale dataset containing triplicated 100 ns explicit-solvent MD trajectories for ~1,300 diverse single-chain proteins.
  • Multi-start Benchmark: A test benchmark curated by the authors consisting of short trajectories ($L=9$ frames) across varying timescales (strides of 128 to 1024 snapshots) from 82 unseen test proteins to evaluate conformation change accuracy.

Key Insights and Ideas

  • Autoregression as a Unifier: The biggest "aha moment" is that flexible frame sequence ordering redefines the dependency structure. By changing what the model conditions on at inference time, a single autoregressive model can seamlessly switch from forward simulation to interpolation or unconditional sampling.
  • Decoupling Time and Space: By separating temporal sequence modeling (Llama layers) from 3D spatial denoising ($SE(3)$ diffusion), the model avoids the complex spatiotemporal denoising mixtures that plague video generation models, leading to more stable structural physics.
  • The Necessity of Hybrid Training: The authors discovered that training purely on trajectories degrades a model's ability to sample independent equilibrium conformations. Mixing trajectory and single-frame objectives is essential to balance kinetic accuracy with distributional diversity.

Key Results

  • Trajectory Simulation: CONFROVER significantly outperformed the flow-based baseline MDGEN, showing a much higher Pearson correlation (0.73 vs 0.18 in PCA 2D space) in capturing the true magnitude of conformational changes.
Figure 4: Visualization of six proteins from multi-start. Trajectory conformations are colored by their secondary structures and superposed to show the dynamic ensemble. MDGEN primarily exhibits local movements, whereas CONFROVER captures conformations changes similar to MD simulations.
  • Time-Independent Sampling: Despite being a multi-purpose model, CONFROVER matched or exceeded the performance of specialized SOTA ensemble generators like ALPHAFLOW and CONFDIFF across multiple distributional accuracy metrics.
  • Conformation Interpolation: CONFROVER-INTERP successfully generated smooth, physically plausible transition pathways between distinct endpoints, whereas the base model without interpolation training failed to progress toward the end state.
  • Massive Speedups: Compared to classical MD simulations using OpenMM, CONFROVER achieved a 14.95× speedup for small proteins (<150 residues) and up to a 52.77× speedup for larger proteins (600–724 residues).

Limitations and Future Work

  • Dataset Scope: ATLAS only contains single-chain proteins, meaning CONFROVER has not yet been tested on massive multi-protein complexes or large-scale functional state transitions.
  • Structural Fidelity Gap: While CONFROVER narrows the gap with classical MD, it still occasionally falls short of full physical fidelity and can miss high-energy barrier transitions.
  • Computational Bottleneck: The triangular updates within the structural modules are memory-intensive, which currently limits the model's scalability to exceptionally long trajectories or massive proteins. The authors suggest exploring more efficient architectures and incorporating explicit energy/force information in future iterations.

Summary updated

Abstract

Understanding protein dynamics is critical for elucidating their biological functions. The increasing availability of molecular dynamics (MD) data enables the training of deep generative models to efficiently explore the conformational space of proteins. However, existing approaches either fail to explicitly capture the temporal dependencies between conformations or do not support direct generation of time-independent samples. To address these limitations, we introduce ConfRover, an autoregressive model that simultaneously learns protein conformation and dynamics from MD trajectories, supporting both time-dependent and time-independent sampling. At the core of our model is a modular architecture comprising: (i) an encoding layer, adapted from protein folding models, that embeds protein-specific information and conformation at each time frame into a latent space; (ii) a temporal module, a sequence model that captures conformational dynamics across frames; and (iii) an SE(3) diffusion model as the structure decoder, generating conformations in continuous space. Experiments on ATLAS, a large-scale protein MD dataset of diverse structures, demonstrate the effectiveness of our model in learning conformational dynamics and supporting a wide range of downstream tasks. ConfRover is the first model to sample both protein conformations and trajectories within a single framework, offering a novel and flexible approach for learning from protein MD data. Project website: https://bytedance-seed.github.io/ConfRover.
BibTeX
@inproceedings{shen2025confrover,
  author = {Yuning Shen and Lihao Wang and Huizhuo Yuan and Yan Wang and Bangji Yang and Quanquan Gu},
  title = {{ConfRover: Simultaneous Modeling of Protein Conformation and Dynamics via Autoregression}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=jj0nJQYFlW},
}
Figure