TL;DR
The authors introduce EpiBench, the first closed-book, sequence-based benchmark designed to evaluate whether Large Language Models (LLMs) can reason about epitopes across the antibody drug discovery workflow. Evaluating nine state-of-the-art LLMs reveals that while they capture coarse antigen properties, they fundamentally struggle with precise antibody-specific grounding, long-context residue localization, and biological coordinate mapping.
What Problem Does This Paper Address?
In antibody drug discovery, knowing where an antibody binds to an antigen—the epitope—is everything. It dictates whether a drug will successfully block a pathogen, trigger the right immune response, or quickly fall victim to mutational escape.
While specialized deep learning models (like BepiPred-3.0) can predict epitopes from structural or sequence data, they are narrow, single-task tools. Conversely, general-purpose LLMs have shown incredible biomedical reasoning skills, but it has remained a mystery whether they can look at raw, unannotated protein sequences and infer complex antibody-antigen interface dynamics.
Existing benchmarks either focus on isolated prediction tasks or rely heavily on 3D structural data. To bridge this gap, the authors present EpiBench to test if LLMs can handle a sequence-driven decision chain simulating a real-world therapeutic antibody development pipeline.
Methods
EpiBench transforms complex biological questions into a sequence-only, closed-book, and automatically scorable format. Crucially, the benchmark strips out all metadata identifiers (such as PDB IDs, antigen/antibody names, and paper metadata) to prevent the models from simply looking up answers from their training data.
As shown in Figure 1, EpiBench evaluates LLMs across five interconnected tasks:
- Targetable Region Discovery (T1): Predicting which specific residue indices on an antigen are likely to be part of an epitope (open-ended residue prediction).
- Antibody-Conditioned Epitope Identification (T2): Matching a specific antibody sequence to its correct epitope from a set of candidates (multiple-choice).
- Antibody Epitope Binning (T3): Determining if two different antibodies bind to overlapping regions on the same antigen (binary judgment).
- Functional Epitope Assessment (T4): Distinguishing epitopes that cause a downstream functional outcome (e.g., neutralization) from those that merely bind (multiple-choice).
- Antibody Escape Assessment (T5): Predicting whether a single point mutation on the antigen will disrupt binding for a specific antibody (binary judgment).
The authors evaluated nine prominent LLMs, including reasoning models (like Gemini-3-Flash, GPT-5.5, and DeepSeek-V4-Pro) and non-reasoning models (like GPT-4o and Grok 4.20), using controlled sampling strategies to eliminate simple shortcuts like sequence-similarity biases.
Datasets
EpiBench comprises 1,609 curated samples derived from four highly trusted experimental sources:
- AsEP & SAbDab: Used for structural antibody-antigen contact data to ground Tasks 1–3.
- IEDB (Immune Epitope Database): Curated functional B-cell assays used for Task 4.
- DMS (Deep Mutational Scanning): Quantitative escape measurements used for Task 5.
Key Insights and Ideas
- Coarse vs. Fine-Grained Understanding: LLMs are decent at identifying general "epitope-like" regions on an antigen, but they fail when forced to ground that prediction against a specific antibody sequence.
- The Long-Context Penalty: As antigen sequences grow longer, an LLM's ability to precisely localize sparse epitope signals plummets. In Task 1, the average recall dropped from 81.3% for short antigens (<200 residues) to a dismal 12.8% for long antigens (>800 residues).
- Chain-of-Thought (CoT) is Inconsistent: While explicit reasoning prompts significantly boosted performance in open-ended region discovery (Task 1), they provided negligible or even negative effects on tasks requiring strict sequence-to-sequence matching (Tasks 2–5).
- The Coordinate Misalignment Flaw: A deep dive into failure modes revealed that LLMs frequently make catastrophic coordinate math errors. For example, a model might correctly recall a specific mutation's biological impact but fail because it incorrectly mapped a local sequence index to a global index.
Key Results
The evaluation showed that no single LLM dominates across all tasks, and overall performance leaves substantial room for improvement:
- Task 1 (Region Discovery): Gemini-3-Flash achieved the highest
RegionRecall@50at 56.9%, but all models hovered near random guessing (~50%) on AUROC, proving they cannot reliably separate epitope from non-epitope residues across an entire sequence. - Task 2 (Epitope ID): Grok 4.20 led with 33.6% accuracy, which is only marginally better than the 25% random baseline.
- Task 3 (Binning): This task saw the highest performance, with GPT-5.5 reaching 67.3% accuracy (against a 50% baseline), likely because it allows pairwise sequence-similarity cues.
- Specialized Models Still Win: Where directly comparable, specialized sequence-based models like BepiPred-3.0 comfortably outperformed general LLMs.
Limitations and Future Work
EpiBench exposes a critical bottleneck: current general-purpose LLMs lack true sequence-grounded structural intuition. They rely heavily on indirect biological proxies—like looking for familiar sequence motifs or applying generic chemical heuristics—rather than modeling the actual physical interface from the sequence.
Furthermore, the benchmark is currently limited to linear and structural representations translated into text strings. The authors suggest that future progress will require developing biomedical LLMs that natively integrate sequence-coordinate tracking or hybrid architectures capable of bridging textual reasoning with geometric protein language models.
Summary updated