When Does Structure Help? The Information Bonus of AlphaFold2 Representations over Protein Language Models

Kargi ChauhanAI4Science @ ICML 2026 · OpenReview

TL;DR

This paper introduces the Information Bonus (IB), a metric to determine if the high computational cost of AlphaFold2 (AF2) structural representations is actually worth it compared to cheaper sequence-only models like ESM-2. The authors find that while sequence models dominate tasks like binding affinity and flexibility, AF2 provides a unique "bonus" for complex tasks like allostery that require understanding long-range 3D communication.

What Problem Does This Paper Address?

In the world of "AI for Science," researchers face a constant trade-off between speed and accuracy. Protein Language Models (PLMs) like ESM-2 can generate embeddings in seconds, while structural models like AlphaFold2 require significant compute and specialized software to infer 3D geometry.

The common wisdom is that "structure matters for structural tasks," but this isn't always true in practice. Many "structural" properties (like whether a region is flexible) are actually encoded in the evolutionary history of the sequence. This paper seeks to turn "which model should I use?" from a vibe-based guess into a measurable engineering decision.

Methods

The authors propose a framework called the Information Bonus (IB). It measures the difference in performance between the best AlphaFold2 representation and ESM-2 on the same task using a frozen linear probe.

  • Frozen Representations: They don't fine-tune the models. Instead, they take the "frozen" internal features (Evoformer representations for AF2 and final-layer embeddings for ESM-2) and train a simple linear model (a "probe") on top. This measures how much information is "linearly accessible" in the model's brain.
  • Protein-Level Splitting: To ensure fair testing, they use GroupKFold cross-validation. This means all residues from a single protein are kept together in either the training or testing set, preventing the model from "cheating" by recognizing a protein it has already seen.
  • The Metric: $IB = Performance(AF2) - Performance(ESM-2)$. A positive IB means structure helps; a negative IB means sequence is enough.

Datasets

The study evaluates the models across three distinct biological regimes:

  1. Binding Affinity (PDBbind): Predicting how tightly a protein binds to a ligand (n=5,680 complexes).
  2. Conformational Flexibility (ATLAS): Predicting Root Mean Square Fluctuation (RMSF)—essentially how much a specific part of a protein wiggles (268 proteins, ~50k residues).
  3. Allosteric-Site Classification (AlloSigDB): Identifying sites where a molecule binding in one spot affects the protein's function elsewhere (47 proteins, ~10k residues).

Key Insights and Ideas

  • Evolution is a Powerful Proxy: ESM-2 often outperforms AF2 because evolution has already "solved" many structural problems. Binding sites and flexible loops leave distinct signatures in protein sequences that PLMs can pick up without ever seeing a 3D coordinate.
  • Allostery is the "Missing Variable": Allostery is the only task where AF2 showed a clear advantage. Because allostery depends on long-range communication through the folded 3D network of the protein, the "geometry-aware" updates in AF2's Evoformer capture signals that sequence-only models simply cannot recover.
  • The Leakage Trap: The authors discovered that "naive" residue-level splits (putting some residues of Protein A in training and others in testing) inflate performance by 27–39%. This happens because the probe learns to identify the specific protein rather than the underlying physics. This error is so significant it can actually reverse which model appears to be better.

Key Results

  • Binding Affinity: ESM-2 wins decisively (IB = -0.141). It achieved a Pearson correlation of 0.449 compared to AF2’s 0.307.
  • Flexibility: ESM-2 wins on binary classification (IB = -0.060; AUROC 0.824 vs 0.764). For continuous regression, AF2 has a tiny, statistically indecisive lead.
  • Allostery: AF2 is the only model to perform better than a random guess (IB = +0.064; AUROC 0.548 vs 0.485).
  • Diagnostic Tool: The authors argue that IB isn't just a leaderboard score; it’s a diagnostic. If a task has a negative IB, your AI system should stick to fast sequence models. If it's positive, the compute "tax" for AlphaFold2 is a necessary investment.

Limitations and Future Work

  • Dataset Size: The allostery dataset is quite small (only 47 proteins), meaning the results there are "directional" rather than definitive.
  • Single-Sequence AF2: The study used AF2 in "single-sequence" mode (no Multiple Sequence Alignment) to keep the comparison to ESM-2 fair. Using full MSA-based AlphaFold2 might increase the structural advantage but would also drastically increase the cost.
  • Linearity: The use of linear probes measures "accessible" info. It’s possible that complex, non-linear "heads" or full fine-tuning could extract more from these models, though that would change the cost/latency profile the authors are trying to measure.

Summary updated

Abstract

AI scientist systems increasingly choose biological foundation models before they choose experiments. In protein pipelines, this creates a concrete engineering and scientific question: when is the cost of structural inference worth paying over a cheaper sequence-only model? We introduce the information bonus (IB), a task-level metric that measures the linearly accessible advantage of frozen single-sequence AlphaFold2 Evoformer representations over frozen ESM-2 embeddings under protein-level cross-validation. Across binding affinity regression (PDBbind, n=5,680), conformational flexibility (ATLAS molecular dynamics, 268 proteins), and allosteric-site classification (AlloSigDB, n=9,925 residues), IB is sharply mechanism-dependent. ESM-2 dominates binding affinity (IB=-0.141; Pearson r=0.449 vs. 0.307) and binary flexibility (IB=-0.060; AUROC 0.824 vs. 0.764; p=0.0017). AF2 single representations give the only above-chance allostery predictions (IB=+0.064; AUROC 0.548 vs. 0.485), revealing long-range geometric signal not recovered from sequence alone. We also identify a residue-level leakage artifact: naive residue splits inflate RMSF performance by 27-39% depending on the representation, enough to reverse representation rankings. These results turn representation selection into a measurable decision for AI-for-science systems.
BibTeX
@inproceedings{chauhan2026when,
  author = {Kargi Chauhan},
  title = {{When Does Structure Help? The Information Bonus of AlphaFold2 Representations over Protein Language Models}},
  booktitle = {AI4Science @ ICML},
  year = {2026},
  url = {https://openreview.net/forum?id=fqiyCRGKJS},
}
Figure