Language-model agents can now run most of a gene-expression study on their own. They download public cohorts, normalize expression matrices, and fit association models. They fail at the step that looks easiest: reading the messy sample annotations, such as "disease state: T2D", "age: 54 yrs", or "tissue: tumor (adjacent)", and turning them into a clean trait column. On the GenoTEX benchmark the agent scores 80.63% on gene data processing but 32.28% on clinical trait extraction. GenoMAS reports 91% versus 32% for the same two stages. Every downstream association test inherits these errors. This note proposes changing the form of the code the agent writes for this one stage. Today the agent writes free-form pandas and regex scripts and debugs them against tracebacks. Instead, it would build a small set of explicit rules ("if a field matches disease state: [X] and X is in {T2D, diabetic}, then trait = 1") by generalizing from a few labeled example strings. The rule set is then run over every sample, and its conflicts and gaps are fed back for repair.
The core bet is that clinical extraction fails mostly through two kinds of silent error. Some samples in one dataset get decided differently from others that should match, and some samples are never covered at all. Neither produces a traceback, so neither is visible to execution feedback. A rule set built from examples makes both kinds of error checkable: every sample is covered by some rule, by an explicit exception, or by nothing, and contradictions are explicit. The rule set also reads like near-English. A biologist without a programming background can audit it, which a 60-line pandas script does not allow.
Core Claim
We claim the following. Replace the clinical-extraction step of a GenoMAS-style agent with a loop that does four things. It induces a compact rule program bottom-up from a handful of labeled metadata strings. It checks the program's coverage and consistency over all samples. It turns uncovered or conflicting samples into targeted queries or explicit exceptions. It rolls back any revision that breaks a checked invariant. This loop should raise clinical trait extraction accuracy on GenoTEX well above the roughly 32% reported for current agents. It should also raise downstream gene identification accuracy, and it should produce an artifact that domain experts can verify faster than code.
The contribution is not "symbolic rules plus an LLM" in the abstract. It is a specific claim that the code under refinement should be a set of rules generalized from examples, rather than imperative code, for the one pipeline stage where tracebacks carry no signal.
Prior Work and Where This Sits
Two systems motivate the gap. GenoTEX built the benchmark and documented the bottleneck. It also reported that repeated review rounds cause "instability", in which agents favor surface instruction-following over sound judgment. GenoMAS splits the analysis into a graph of subtasks it calls Action Units, reviews code with an LLM Code Reviewer, and backtracks upstream when a statistical check fails. It reaches 98.78% bug-free execution, yet clinical extraction stays at 32%. That combination is the clearest sign that execution feedback is not the missing signal for this stage. CellAgent shows the same design pattern in single-cell work: a planner, an executor, and an evaluator that judges outputs by metrics and by looking at plots. None of these three systems represents its decisions as checkable rules. We keep their agent structure and change only the representation used for one stage.
Two symbolic learners supply the method. Learning Symbolic Rules for Reasoning in Quasi-Natural Language introduces MetaInduce. It proposes candidate rules by anti-unification, meaning it finds the most specific template two sentences share, with their differing parts replaced by variables. It then selects a compact, consistent rule set with a weighted MAX-SAT solver, which picks the subset of candidate rules that best satisfies weighted constraints. MetaInduce reaches 99.7% on ParaRules but has only been tested on synthetic and linguistic benchmarks. We reuse its propose-then-select design on GEO characteristic strings, which are close to the "quasi-natural language" it was built for. Learning Brave Assumption-Based Argumentation Frameworks via ASP contributes assumption introduction. When a rule is over-general, the method attaches an assumption whose contrary lists the exceptions, instead of failing. This matters for metadata, where "control" usually means trait = 0 except in datasets where it labels a treatment arm. The same paper also constrains us. It reports that ILASP, a classical inductive logic programming system, returns unsatisfiable on acute and times out on breast-w, and that its own folding search is unguided and scales poorly. We therefore keep the rule language small (field templates, value sets, and exceptions) and keep the problem per-dataset, where sample counts are in the hundreds.
The iterative-refinement literature supplies the loop discipline. MEIC reverts Verilog code to the last stable version when a fix causes regressions. It names the "new error trap", where fixing one bug introduces another. We apply rollback, but the invariants are checked on the rule program rather than on simulator output. CAAFE is the nearest bridge between rules and code. An LLM writes tabular feature code, a model scores it, and only code that helps is kept. CAAFE refines isolated snippets judged by predictive gain and never induces rules. Our acceptance test is logical coverage and consistency, not downstream accuracy, which avoids selecting labels that fit the outcome.
Research Program
The system works dataset by dataset inside the GenoMAS pipeline, using its open codebase Liu-Hy/GenoMAS. The only change is the clinical-trait Action Unit.
The first step turns each sample's characteristic fields into ground facts, meaning plain observations of the form field-key and value string per sample. Anti-unification groups these strings into templates, so that "age: 54 yrs" and "age: 61 yrs" merge into "age: [N] yrs".
The second step chooses examples. The LLM labels one or two representative strings per template for the requested trait, which is far fewer than labeling every sample. This follows Ling's observation in Inductive Learning from Good Examples that a few well-chosen examples can make induction tractable and resistant to noise. The LLM also states its domain assumptions in plain text, for example "treat 'adjacent normal' as control".
The third step is induction. Candidate rules come from generalizing the labeled examples, and MAX-SAT selects the smallest rule set that agrees with all labels. Rules that are over-general get assumption-based exceptions instead of being dropped.
The fourth step is the refinement loop. The rule program compiles to a deterministic pandas function and runs on all samples. A checker then computes uncovered samples, samples where two rules fire with different values, trait values outside the declared domain, mismatches with the expression-matrix sample IDs, and group sizes that contradict the series description, such as "20 cases, 20 controls". Each violation becomes one targeted query to the LLM about that specific string, and the answer is added as a new labeled example. Induction then reruns. A revision is kept only if it reduces the violation count and keeps every earlier label satisfied. Otherwise the system rolls back, as MEIC does. The loop stops when no violations remain or when a fixed query budget runs out. Any samples still unresolved are marked missing instead of guessed.
The final rule set is rendered as sentences, for example "Samples whose 'disease state' is T2D or diabetic are cases, except those whose 'treatment' is metformin". This rendering is the human-facing output.
Before building any of this, we will classify GenoMAS's clinical-extraction failures on a stratified sample of GenoTEX problems. The categories are wrong field chosen, inconsistent value mapping, missed samples, wrong coding of the trait, and trait not recoverable from metadata. If most errors fall in the first or last category, induction cannot help, and we would redirect effort before building the system.
Baselines
The naive reference point is GenoMAS run unchanged, with its LLM-written extraction code and its Code Reviewer. It is the strongest existing system on this exact task and shares every other component with our system, so any difference comes from the clinical stage. The reported numbers are 32% clinical phenotype extraction accuracy and 60.48% gene identification F1 (GenoMAS). GenoAgent's 32.28% clinical CSC and 0.74 AUROC (GenoTEX) serve as a second reference. We rerun GenoMAS with identical model backbones, the same problem split, and the same cap on LLM tokens per dataset. The cap matters because our method makes more, smaller LLM calls. Beating GenoMAS means a clear gain in clinical trait CSC, which we target at a doubling toward the 80% range seen for gene processing. Downstream gene identification F1 must also improve, not merely stay flat.
The second baseline labels every sample directly. Each sample's characteristic string goes to the same LLM with the trait definition, and the outputs are concatenated. This is the strongest simple competitor, because it needs no code and no rules. We give it the same token budget and also report it with unlimited budget.
The third baseline is an ablation that isolates induction. Per-row LLM labels pass through the same coverage and consistency checker and the same targeted re-queries, but nothing is generalized into rules. If this ablation matches the full system, then the value lies in the checker and not in rule induction.
A second ablation removes assumption-based exceptions and keeps only strict rules. It measures whether defeasible rules matter on real metadata.
The last comparison replaces our induction engine with ILASP under the same rule language. This tests whether ordinary inductive logic programming suffices.
Why not just…?
Why not just have the LLM label every sample directly? This is the most serious threat. GEO strings are short, and frontier models read them well. Nothing in the file rules it out, since no reported result on Lacuna isolates per-row labeling on GenoTEX clinical traits. The case for rules rests on three things. Per-row labeling can map identical strings differently across calls. It has no notion that a sample was skipped. Its output cannot be audited without re-reading every row. Baselines two and three settle the question directly. If per-row labeling plus the consistency checker matches the full system on accuracy, we will report that the checker is the contribution. The rule representation would then be justified only by audit time in the user study, and the claim shrinks accordingly.
Why not just add more review rounds or a stronger reviewer to GenoMAS? GenoTEX reports that extended review loops in this domain show diminishing returns and instability. GenoMAS's own high execution success alongside 32% extraction suggests the reviewer cannot see these errors. We will still run GenoMAS with double its review budget as a check. If more review closes the gap, our proposal is unnecessary.
Why not just use a classical inductive logic programming system? ILASP returned unsatisfiable on acute and timed out after 15 minutes on breast-w, whereas ASP-ABAlearn_B finished in 16.32 seconds (Learning Brave Assumption-Based Argumentation Frameworks via ASP). Classical systems also need predicates written by hand beforehand. The ILASP comparison above tests whether this holds on metadata, where the problems are much smaller.
Why not just ask the LLM to write extraction code plus its own unit tests? Tests written by the same model encode the same misreadings of the data. We include this as a secondary variant of GenoMAS under the same budget.
Proposed Experiments and Decisive Checks
The main experiment runs all systems on the GenoTEX clinical extraction stage and on the full gene–trait association pipeline. We report clinical CSC, gene identification F1, and AUROC against the human 0.89 reference. Results are stratified by the failure categories from the diagnostic step, so that gains can be attributed to error types induction should fix, namely inconsistency and missed samples, and not to the others.
Two checks are decisive. The full system must beat the per-row-plus-checker ablation; if it does not, the induction claim fails. Gains must also concentrate in the inconsistency and missed-sample categories; if gains instead appear uniformly, the mechanism is not what we claim.
A transfer test asks whether a rule program induced on one GEO series for a trait reduces the number of queries needed on other series for the same trait. Free-form scripts cannot offer this property.
A small user study gives bench biologists either the rendered rules or the GenoMAS script for the same dataset, with planted errors. We measure time to verify and the fraction of planted errors found.
We will also report the number of LLM queries per dataset, wall-clock time, and how often rollback fires. A high rollback rate would show that the loop guards against the new-error trap in this setting rather than merely asserting that it does.
References
- GenoTEX: A Benchmark for Evaluating LLM-Based Exploration of Gene Expression Data in Alignment with Bioinformaticians
- GenoMAS: A Multi-Agent Framework for Scientific Discovery via Code-Driven Gene Expression Analysis
- CellAgent: LLM-Driven Multi-Agent Framework for Natural Language-Based Single-Cell Analysis
- Learning Symbolic Rules for Reasoning in Quasi-Natural Language
- Learning Brave Assumption-Based Argumentation Frameworks via ASP
- MEIC: Re-thinking RTL Debug Automation using LLMs
- Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineering
- Inductive Learning from Good Examples