Generative models that design proteins — choosing both the 3D shape and the amino-acid sequence jointly — work well up to roughly 300 residues and then fall apart: on the standardized evaluation in ProteinBench, most backbone design models lose more than 20% of their self-consistent structure score when going from 300 to 500 amino acids. The proximate cause is memory, not ideas. The network components that reason about geometry — "triangle updates," which check every triple of residues for consistency the way the triangle inequality constrains three distances — need a table covering every pair of residues (size grows with the square of protein length) and update it over all triples (work grows with the cube). La-Proteina reached 800 residues only by deleting these layers outright, trading local geometric precision for global length capacity.
Compilers solved a structurally identical problem decades ago. When a compiler cannot prove that two memory accesses target different locations, it must assume they collide and serialize them. Techniques like pointer disaggregation — splitting a pointer into a fixed container handle plus an explicit integer offset — turn opaque address arithmetic into affine expressions a compiler can prove independent, yielding an 11x speedup on OpenSSL PBKDF2 where standard compilers gave up. This proposal imports that way of thinking into protein generation: treat the residue-pair table as memory, run conservative static dependency analysis over it, and prove which parts of the geometric reasoning can be computed independently or skipped entirely — soundly, without changing what the model computes.
Core Claim
The residue-pair table of a protein generative model can be lifted into an explicit dataflow representation where its two roles — a table of memory locations indexed by residue pairs, and a record of physical proximity between residue pairs — are disaggregated. On that representation, a conservative static analyzer can (a) prove that blocks of triangle updates write to disjoint table regions and therefore run out-of-order or on separate devices, and (b) prove that residue pairs whose certified spatial bounds cannot overlap need no interaction term at all, keeping the term only when the proof fails. The analyzer is conservative in the compiler's sense: whenever it cannot prove independence, it assumes dependence and keeps the computation. The claim is that this restores full triangle-based geometric reasoning at 500–800 residues within the memory envelope that currently forces models to delete it, cutting the documented 300-to-500-residue quality cliff substantially.
Research Program
The first step is a lift, in the spirit of DaCe-style dataflow lifting and lifting C semantics for dataflow optimization, which bypass opaque pointers by rewriting programs as explicit dataflow graphs. We take a long-chain generator of the La-Proteina type — an explicit backbone coordinate track plus per-residue latent vectors, trained by flow matching, i.e., learning to gradually denoise random noise into a protein — and rewrite its pair representation as a first-class array $P \in \mathbb{R}^{L \times L \times C}$, where $L$ is protein length and $C$ the pair-feature dimension, with triangle updates exposed as array computations rather than fused neural-network layers. Concretely, a triangle update has the form
$$P'(i,j) = f\!\left(P(i,j),\ \textstyle\sum_k g(P(i,k), P(j,k))\right),$$
which needs $O(L^2 C)$ storage and $O(L^3 C)$ work. Blocking residues into groups of width $B$ and computing one output block at a time requires only a row band and a column band resident, so peak memory falls from $O(L^2 C)$ to $O(BLC)$ — for $L = 800$ and $B = 64$, roughly a 12-fold reduction.
The analyzer then does two jobs. The scheduling job is exact: because each output block writes a disjoint region of $P'$, the analysis proves block independence and emits a schedule with minimal data movement across the memory hierarchy, following the data-movement-aware partitioning of Tang et al.; the partitioned schedule is bit-equivalent to the dense one by construction. The pruning job is where the disaggregation idea from Iterating Pointers earns its keep: the pair index $(i,j)$ simultaneously names a table location and a spatial fact about residues $i$ and $j$, and separating those roles lets the analyzer reason about physical overlap. During sampling, each residue's backbone position carries a certified radius $r_i$ derived from the integration error bound of the flow-matching trajectory; a pair term is skipped only when
$$\|x_i - x_j\| > r_i + r_j + d,$$
where $d$ is the support of the local geometric kernel, and kept otherwise. Long proteins are mostly multi-domain, so the provably-far fraction should be large exactly when $L$ is large. We are honest about the boundary of soundness, in the same way LLM-Vectorizer openly reports proving only 38.2% of loops formally: the exclusion proof is exact for compactly supported kernels (local geometry and gated triangle-multiplication terms) and approximate for softmax attention, where we measure the deviation empirically rather than claim a proof we cannot deliver.
The secondary ingredient is a cheap symbolic guardrail: a static stereochemistry checker (bond lengths, Ramachandran angle bounds, certified steric non-overlap from the same spatial bounds) that runs before any folding evaluation. This targets a documented evaluation bottleneck — the FrameFlow codebase notes each benchmark run requires over 2600 ProteinMPNN and AlphaFold2 evaluations, which the authors call "very arduous" — by filtering candidates that provably violate physical bounds before the expensive fold.
Baselines
The do-nothing reference is the unmodified dense architecture with a full pair table and triangle updates. It is the correctness oracle: at lengths where it fits in memory (roughly 250–300 residues on our hardware), the partitioned schedule must reproduce its outputs exactly, and where it runs out of memory it defines the failure mode we are attacking. No reported result found on Lacuna for a dense triangle-update model generating beyond roughly 500 residues; the documented evidence is that La-Proteina hit out-of-memory failures and deleted the layers instead.
The strongest existing competitor is La-Proteina, the only published system generating valid all-atom proteins to 800 residues, with 75.0% all-atom co-designability against a previous best of about 36.7%, and 21–25 of 26 difficult motif-scaffolding tasks against 4 for the closest competitor. We compare under matched conditions: same sampling budget per task, same folding evaluator, and either the published numbers or a reproduction under matched compute, and we beat it only if we match its long-chain reach while improving local geometric quality (long-chain self-consistent scores, steric validity) thanks to restored triangle reasoning. On the 24-motif scaffolding benchmark we also compare against FrameFlow, which generated 353 unique structural clusters versus RFdiffusion's 141 and solved 20 of 24 motifs in roughly 18 seconds per 100-residue backbone; its released code is also our implementation substrate, which keeps the comparison architecturally fair.
Two ablations isolate the key ingredient. First, the same blocked model with the dependency analysis disabled — every block serialized, no data-movement minimization — separates the gain from proven independence versus mere blocking. Second, uniform random sparsification of pair terms at matched FLOPs, with no spatial certificates, tests whether certified exclusion rather than plain sparsity is what preserves quality; ProteinInvBench showed in the inverse-folding setting that cheap shortcuts that look fine on one metric (sequence recovery) fail on self-consistent structure, so we evaluate this ablation on the same multi-metric footing.
Why not just…?
Why not just use memory-efficient attention kernels or hand-tiled implementations? This is the most serious alternative, and we treat it as such: FlashAttention-style kernels already tile attention, and a strong engineer could hand-block the triangle updates. Two things distinguish the proposal. Empirically, off-the-shelf efficiency did not solve the problem — La-Proteina worked within standard PyTorch tooling and still had to delete the layers at 800 residues, and the redundant-transfer pathology that conservative alias analysis causes in GPU offloading, documented by Marzen et al., is precisely the failure mode of naive tiling across devices. Conceptually, our contribution is the analysis, not the tiling: the certified spatial exclusion (which kernels do not provide) and the soundness guarantee. The decisive experiment is a best-effort hand-tiled baseline built by a strong engineer under the same budget; if it matches our quality and memory at 500–800 residues, the scheduling half of the claim is wrong and we will say so — but the pruning half, which kernels cannot replicate, stands or falls on the matched-FLOPs sparsity ablation.
Why not just compress harder, as La-Proteina already did? Latent compression is the incumbent answer and a real threat: its published numbers are strong. But ProteinBench attributes the 300-to-500-residue cliff precisely to the absence of high-order geometric reasoning, and La-Proteina's own account confirms it deleted that reasoning to fit memory. Our experiment directly tests whether reinstating triangle updates at long lengths closes part of that cliff without giving up the 800-residue reach; if the cliff persists with triangle reasoning restored, the compression route wins and the hypothesis is falsified.
Why not just buy more hardware or use naive model parallelism? More GPUs trivially fit the dense table; the question is whether static analysis earns its keep at fixed compute. Tang et al. show that data-movement-aware partitioning dominates naive splitting for exactly these dense access patterns, and Lepori et al. show that standard tools leave an 11x gap on the table when analysis is too conservative. We run the multi-GPU naive-parallel baseline at the same total FLOP budget and report memory, communication volume, and wall-clock; the claim is only interesting if it wins there.
Why not just keep verifying everything with AlphaFold2 afterward? Post-hoc folding remains the gold standard and we use it for all headline metrics. The symbolic guardrail is a pre-filter, not a replacement: it exists because the FrameFlow codebase documents that evaluation alone consumes over 2600 ProteinMPNN and AlphaFold2 runs per benchmark. The guardrail counts as useful only if it prunes a meaningful fraction of candidates, and its pruning decisions agree with the folding verdicts on the survivors; we measure that agreement directly.
Proposed Experiments
The main study is a length sweep from 100 to 800 residues on backbone and sequence-structure co-design, following the ProteinBench protocol with self-consistent structure scores computed by the same folding models, in length bins that bracket the documented cliff. The headline number is the 300-to-500-residue drop in self-consistent score: ProteinBench reports a drop above 20% for current models, and the proposal succeeds if the certified-exclusion model keeps that drop under about 10% while staying inside the single-node memory envelope where the dense model fails. Quality is cross-checked on motif scaffolding using both the 24-motif benchmark of FrameFlow and La-Proteina's 26-task suite, reporting solved tasks and unique structural clusters against the published 20/24 and 21–25/26 figures at matched sampling budgets.
Soundness is checked three ways. At lengths where the dense model fits, the partitioned schedule must reproduce its outputs exactly (maximum absolute deviation of zero), which validates the tile-independence proof. The spatial pruning fraction and its incurred approximation error are reported as a function of length and sampling time, since provably-far pairs should dominate exactly at long lengths. And the symbolic guardrail is evaluated by pruning-then-folding: we report how many expensive folding evaluations are skipped and whether the final model rankings change, which is the only test that matters for a filter.
Efficiency closes the loop: peak memory, communication volume, and wall-clock per generated protein as functions of $L$, for the dense model (until it fails), the La-Proteina configuration, the serialized-blocking ablation, the matched-FLOPs random-sparsity ablation, and the full analyzer. The result that would make the paper is a curve showing full triangle reasoning sustained to 800 residues at the memory cost today's models pay for their stripped-down versions — the compiler's old bargain, soundness first and speed as a proof, applied to a place nobody expected it.
References
- ProteinBench: A Holistic Evaluation of Protein Foundation Models
- La-Proteina: Atomistic Protein Generation via Partially Latent Flow Matching
- Iterating Pointers: Enabling Static Analysis for Loop-based Pointers
- Lifting C Semantics for Dataflow Optimization
- Data movement aware computation partitioning
- LLM-Vectorizer: LLM-Based Verified Loop Vectorizer
- Improved motif-scaffolding with SE(3) flow matching
- ProteinInvBench: Benchmarking Protein Inverse Folding on Diverse Tasks, Models, and Metrics
- Static Generation of Efficient OpenMP Offload Data Mappings