Cryo-electron microscopy (cryo-EM) produces a three-dimensional "density map", a voxel grid showing where a protein's electrons are. Turning that map into an atomic model is increasingly done by neural networks that read the 3D map and the protein's amino-acid sequence together. When the map is sharp, the density alone pins down the atoms. When it is blurry, at 4 to 10 Å resolution where side chains disappear, the network can lean on the sequence instead. A sequence encoder trained on the heavily duplicated Protein Data Bank (PDB) may simply recall what a familiar protein family looks like and draw it into the map. None of the current systems can tell us which of these is happening. They train and test on map collections that were never filtered for evolutionary relatives.
This note proposes to measure this jointly as a function of map resolution and of the test protein's sequence identity to anything seen in training. We will build a benchmark that crosses those two axes, add a test that catches residues built where there is no density, and test one simple training change aimed at the failure we expect. That change is to sample training maps so that no protein family dominates, and to teach the model that a sequence has to be confirmed by the density rather than trusted. The import is that sequence-identity filtering, routine in sequence-only biological machine learning, has not yet been applied to volumetric structure models.
Core Claim
We claim that much of the advantage that sequence-conditioned density models show at low resolution is recall of homologous training families, not reading of the map. The prediction is specific. On test maps at 4 to 8 Å, the accuracy gap between a sequence-conditioned model and a sequence-free one should shrink sharply when test proteins share less than 30% sequence identity with the training set. That low-identity range is the "twilight zone", where relatives can no longer be reliably detected from sequence alone.
The same models should also place confident residues into regions of the map where density has been deliberately removed, and they should do so more often for familiar families. If neither effect appears, that is a clean and useful negative result: it would mean current builders generalize across families. If the effects do appear, we test whether family-balanced training combined with mismatched-sequence negatives closes the gap.
Grounding in Prior Work
The motivation comes from two sides. On the volumetric side, four systems condition on sequence or learn priors from uncurated corpora:
- ModelAngelo couples a 3D convolutional network with a graph network that reads ESM-1b protein language model embeddings. It trains on about 3,200 map-model pairs from the Electron Microscopy Data Bank (EMDB) and uses a temporal split.
- E3-CryoFold links a 3D transformer to an ESM-2 sequence encoder. It pretrains on 163,284 maps simulated from PDB coordinates and fine-tunes on 7,389 experimental Cryo2StructData maps.
- CryoACE samples density at predicted atom positions inside a guided diffusion model. It weights density against sequence priors using a local resolution estimate.
- CryoFM learns a density prior from 3,479 EMDB maps selected only by a 3.0 Å resolution cutoff.
None of these applies sequence-identity clustering, so none can separate map-reading from family recall. CryoACE's resolution-dependent weighting is the closest existing mechanism to what we propose. However, it decides how much to trust the sequence from map sharpness alone, not from whether the sequence is consistent with the density. CryoDomain went the other way and dropped sequence entirely at low resolution, reporting robustness to 10 Å. Its retrieval database built from CATH and SCOPe is also not filtered for superfamily overlap. We use it as the sequence-free comparison point rather than as a competitor on model building.
On the sequence side, the 35–40% identity cutoffs used by Han et al. and Zhao et al. (DDIG-in) supply the method we reuse. These papers do not themselves supply evidence that the volumetric models fail; that part is our hypothesis. Two more papers constrain the design:
- S²Drug found that downweighting homologous families during pretraining helped under homology-excluded evaluation, which is the closest precedent for our sampling scheme. It also reports that extreme loss-balancing weights destabilize training, a warning we take seriously for our negative-sequence term.
- ABAG-docking shows that sequence identity and structure can come apart: entries 6ZDG and 6ZER share high sequence identity but differ structurally. Clustering by sequence alone could therefore throw away genuinely different conformations, so we test a combined sequence-and-structure clustering as an ablation rather than assuming it.
Research Program
The first component is the benchmark. Starting from Cryo2StructData, we cluster all chains by sequence at 30%, 40% and 95% identity. We then hold out entire clusters so that every test chain has a known maximum identity to any training chain.
Resolution has to be varied independently of protein family, because in experimental data the two are tangled. We therefore also simulate maps from held-out PDB structures, using the Gaussian-blur projection that E3-CryoFold uses, at 3, 4, 6, 8 and 10 Å. This produces a grid of resolution against maximum training identity, with experimental maps as a check on realism.
The second component is a hallucination probe. For each test map we erase density over a contiguous segment of 10 to 30 residues, simulating a disordered loop. We then count residues the model builds inside the erased region with high confidence, along with their agreement with the true structure. Confident placement into empty space can only come from a prior. Agreement with the held-out true structure there is suspicious when the family was seen in training and impossible to obtain from evidence otherwise. We pair this with a sequence-swap test: we give the true map with a length-matched sequence from an unrelated protein, and measure how far the output structure moves toward that protein's known fold.
The third component is the intervention, applied to a dual-modality builder that we retrain at reduced scale. Training batches sample clusters uniformly, then chains within a cluster, at the 30% threshold. Each map is additionally paired, with some probability, with a mismatched sequence drawn from a different cluster. For these pairs, the model's per-residue confidence is trained toward zero while its density-only backbone prediction is still supervised. The intended effect is that the model treats the sequence as a hypothesis the map must support.
Writing $\mathcal{L}_{\text{build}}$ for the usual coordinate and sequence-assignment loss and $\mathcal{L}_{\text{mis}}$ for the cross-entropy pushing confidence to zero on mismatched pairs, the objective is
$$\mathcal{L} = \mathcal{L}_{\text{build}} + \lambda\, \mathcal{L}_{\text{mis}},$$
where $\lambda$ is a single weight. Following the S²Drug warning, $\lambda$ will be swept over a small grid and selected on a validation set of held-out clusters.
Baselines
Naive reference. The same architecture is trained on the uncurated corpus with the temporal or random split these papers use. Its gap between the in-distribution test set and the twilight-zone test set is the quantity we expect to be large. Beating it means a smaller gap at equal or better twilight-zone accuracy.
Strongest published system. This is E3-CryoFold, which reports an average TM-score of 0.815 on Cryo2StructData, against 0.20 for Cryo2Struct. TM-score is a 0-to-1 measure of overall fold agreement. Retraining it on 163,284 maps is beyond a fair matched budget. We will therefore re-score its released outputs or checkpoints, if available, on our cluster-stratified test sets, and treat them as a contaminated-but-strong reference whose training set we cannot clean. Separately, we will train a reduced replica on our clean splits.
ModelAngelo. It is the established sequence-conditioned builder. The file gives only qualitative results: sequence recall comparable to manual expert building on maps better than 3.5 Å. We found no scalar result on Lacuna for ModelAngelo on Cryo2StructData. It will be run the same way as E3-CryoFold.
Sequence-free control. This is our replica with the sequence encoder removed, in the spirit of CryoDomain. It tells us how much of any sequence advantage survives on novel families.
Ablations. These isolate the key ingredients: cluster-balanced sampling alone, mismatched-sequence negatives alone, and sequence-only versus sequence-plus-TM-score clustering.
All replicas share data, architecture, step count and an equal hyperparameter-search budget. The intervention counts as a success if, at 4 to 8 Å and below 30% identity, it raises TM-score and sequence recall over the naive replica beyond the variation across seeds, roughly halves confident building in erased density, and does not lose more than a small margin on in-distribution maps.
Why not just…?
Why not just re-split the test set with sequence clustering and stop there? This is a real threat to the training-side contribution, and it is why the benchmark is the primary deliverable. If re-splitting alone shows a small homology gap, the intervention is unnecessary and we will say so. The intervention is justified only if the gap is large and cluster-balanced sampling alone fails to close it. That case is tested directly in the ablations.
Why not trust the temporal splits already used? A map deposited after a cutoff date can still be a close relative of training proteins; viral spikes are a likely example. Whether this happens in ModelAngelo's 28-structure test set is unknown to us. Our first measurement is the identity distribution of existing temporal test sets against their training sets, which settles the question empirically.
Why not drop the sequence at low resolution, as CryoDomain does? CryoDomain shows that sequence-free retrieval is robust to 10 Å. However, it identifies domains rather than building atomic models, and it gives up sequence information that may genuinely help when the density supports it. The sequence-free control measures exactly what is lost. If the sequence-free control matches the conditioned model below 30% identity, that supports CryoDomain's choice and we would report it.
Why not rely on CryoACE-style resolution weighting? Resolution tells the model how blurry the map is, not whether the sequence's implied fold matches it. The sequence-swap test distinguishes the two. A resolution-weighted model should move toward the decoy fold just as much at low resolution, whereas a mismatch-trained model should not.
Decisive Checks
The central check is the resolution-by-identity grid of TM-score for the naive conditioned replica minus the sequence-free replica. The hypothesis predicts that this advantage concentrates in the high-identity, low-resolution corner and falls toward zero in the twilight-zone, low-resolution corner. A flat grid falsifies the claim.
The second check is the erased-density probe. Confident residues built into empty density, broken down by training identity, should fall for the intervention model at no more than a minor cost to in-distribution TM-score.
The third check addresses the 6ZDG/6ZER problem. It compares sequence-only against sequence-plus-structure clustering on conformationally variable test cases, such as the integrin data in EMPIAR-10345, and asks whether sequence-only clustering discards conformations the model then fails to build.
References
- ModelAngelo: Automated Model Building in Cryo-EM Maps
- One-shot Cryo-EM Complex Structure Determination with High Accuracy and Ultra-fast Speed.
- CryoACE: An Atom-centric Framework for Accurate and Automated Model Building in Cryo-EM
- CryoFM: A Flow-based Foundation Model for Cryo-EM Densities
- CryoDomain: Sequence-free Protein Domain Identification from Low-resolution Cryo-EM Density Maps
- Predicting Ion Channels Genes and Their Types With Machine Learning Techniques
- DDIG-in: discriminating between disease-associated and neutral non-frameshifting micro-indels
- S²Drug: Bridging Protein Sequence and 3D Structure in Contrastive Representation Learning for Virtual Screening
- ABAG-docking benchmark: a non-redundant structure benchmark dataset for antibody-antigen computational docking