Automated microscopy can now image a whole mouse brain in days. Turning those images into complete single-neuron wiring diagrams still takes weeks of human correction per neuron. Most remaining errors are "merges," where two neighbouring axons are wrongly fused, and "splits," where one axon is wrongly broken into pieces. Machine learning has made splits fairly tractable. Merges remain stubborn: the best multimodal language model on a recent proofreading benchmark detects them at roughly 62% accuracy, barely above chance. This proposal argues that the difference is about time, not about how good the images are. A wrong split can be judged by looking at two adjacent fragments. A wrong merge often looks locally plausible and only becomes implausible after several more fragments have been attached, for example when two crossing axons produce an impossible branching pattern three junctions later.
Deep reinforcement learning (RL) is the study of agents that learn to make sequences of decisions from a reward signal. It offers a precise tool for testing this claim: the effective horizon, meaning how many steps of lookahead an agent needs before simple greedy choices become optimal. We propose to cast fragment assembly as a sequential decision problem and measure its effective horizon decision by decision on ground-truth connectomes. We then build a tracer whose lookahead depth matches what the measurement says is needed. The measurement is the main contribution. It either justifies sequential planning for the specific junctions where current pairwise methods fail, or it shows that RL is unnecessary for this problem. Either outcome is useful.
Core Claim
We hypothesize that the decisions involved in assembling an axon from fragments have a bimodal horizon. Most continuation and split-repair decisions can be solved with a single step of lookahead. A minority of decisions at dense fiber crossings need three or more steps. Those few long-horizon decisions account for a disproportionate share of residual merge errors. If this holds, a tracer should be shallow almost everywhere and deep only where needed. It should learn a value function over whole partial reconstructions rather than over fragment pairs.
Grounding in Prior Work
Two lines of tracing work motivate the hypothesis by succeeding at local decisions and struggling at global ones.
Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing trains a contrastive 3D image embedding fused with point-cloud shape features to score fragment pairs. It reaches an F1 of 0.952 on its FlyTracing benchmark. Every decision it makes is pairwise and greedy, and its authors name "reasoning about multiple potential connection candidates simultaneously" as future work. ConnectomeBench shows the same asymmetry in language models: 75–85% accuracy on split correction but about 62% on merge identification.
The strongest system addresses the global problem with search rather than learned sequential decisions. Accelerating Neuron Reconstruction with PATHFINDER stacks three stages. Flood-filling segmentation produces fragments, a segment expander proposes candidate joins, and a point-cloud plausibility scorer filters fragment assemblies during a heuristic combinatorial search. It reaches 94.2% normalized expected run length on axons, along with tenfold fewer merges. PATHFINDER supplies our strongest comparison, and its assembly-level scoring is the design we most need to beat or match more cheaply. NEURD supplies a constraint: hand-written graph rules, such as cutting branches that double back at acute angles, clean merges well. NEURD raises excitatory synapse precision from 0.13 to 0.87. These rules only cut and never join. An unsupervised map of excitatory neuron dendritic morphology shows that morphology varies continuously, which warns against fixed geometric cutoffs.
The method comes from The Effective Horizon Explains Deep RL Performance in Stochastic Environments. That paper shows that in many environments, estimating the value of a simple base policy and acting greedily over a short lookahead is enough. Over half of its 155 benchmark environments are solvable with one step of lookahead. Its algorithm, SQIRL, exploits this. We reuse the definition and the algorithmic recipe, applied to a domain where nobody has measured the horizon.
Two RL-adjacent works are deliberately not our template. Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning uses RL only to choose image masks during pretraining, so it makes no tracing decisions. Recursive Reinforcement Learning handles unboundedly nested tasks. Axonal branching looks recursive, but each subtree in fragment assembly is finite. We would adopt recursive machinery only if the horizon measurement showed that sibling branches interact through long chains, which we consider unlikely.
Research Program
The decision problem. Each episode starts from a seed fragment taken from an existing over-segmentation. The state is the current partial assembly: a point cloud of all attached fragments plus per-fragment image embeddings, together with a frontier of candidate fragments proposed near open endpoints. The agent can take one of four actions:
- attach a frontier candidate;
- reject it;
- detach the most recently attached fragment, which allows backtracking;
- stop.
The reward comes from proofread ground truth. Attaching a correct fragment earns its path length. Attaching a wrong fragment incurs a penalty proportional to the length it adds.
Measuring the horizon. Because ground truth is known, the optimal action at every state is known exactly. For a base policy $\pi$ we estimate its value $Q^\pi$ by Monte Carlo rollouts. We use two base policies: uniformly random, matching the original definition, and pairwise-greedy using the contrastive embedding, which is the realistic one. We then define the $k$-step lookahead action
$$a_k(s) = \arg\max_{a} \max_{a_2,\dots,a_k} \mathbb{E}\Big[\textstyle\sum_{t=1}^{k-1} r_t + Q^\pi(s_k, \cdot)\Big],$$
where $s$ is the current state, $r_t$ the rewards along the lookahead path, and $s_k$ the state reached after $k$ steps. The decision's horizon $H(s)$ is the smallest $k$ for which $a_k(s)$ equals the ground-truth optimal action. We report the distribution of $H$ stratified by four factors: decision type (continuation, split repair, merge rejection), local fiber density, and the presence of imaging artifacts.
The tracer. Following SQIRL, we fit a neural critic that predicts $Q^\pi$ for whole partial assemblies. It takes the point cloud and embeddings of the full assembly, not a fragment pair, as input. The critic is trained by regression on rollout returns, with no bootstrapped deep RL. A cheap classifier, trained on the measured $H$ labels, predicts whether a junction is long-horizon. The tracer then uses one-step lookahead at junctions predicted to be short and $k$-step lookahead elsewhere, where $k$ is the empirical value from the measurement.
Baselines
The naive reference point is the input over-segmentation with no agglomeration, plus nearest-endpoint joining under a distance threshold tuned on the validation split. This bounds how much any method adds.
The strongest pairwise competitor is the multimodal contrastive scorer of Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing, with its reported FlyTracing F1 of 0.952 and an 8.1% (28.8 µm) expected-run-length gain in the mushroom body. We will retrain it on our splits with its published architecture so that candidates and data are identical. Beating it means a statistically significant reduction in merge rate at equal or better split recall, not merely a higher pairwise F1, which is already near its ceiling.
The strongest assembly-level competitor is a PATHFINDER-style search (PATHFINDER). It combines beam search over assemblies using the same pairwise scores with a point-cloud plausibility scorer trained to separate true from false assemblies. On its own mouse cortex volume it reports 94.2% axon nERL. We have not found a reported FlyTracing number for it on Lacuna, so we will reimplement it with the same candidate generator, the same backbone sizes, and a matched budget of plausibility evaluations per decision. Beating it means comparable error at substantially fewer assembly evaluations, or lower error at the same budget.
Two further baselines isolate the key ingredient. The first is full deep RL, using proximal policy optimization (PPO) on the same decision problem, with the same network and a matched number of environment steps. This tests whether genuine RL exploration adds anything beyond shallow lookahead. The second is an ablation that keeps everything fixed but uses $k=1$ everywhere and replaces the assembly critic with the pairwise score. This tests whether the gains come from depth, from assembly-level value, or from both.
Tuning budgets are equal across all learned methods, measured in GPU-hours and hyperparameter trials.
Why not just…?
Why not just run beam search over the pairwise scores? This is the most serious threat. Search over assemblies with a good plausibility scorer is essentially what PATHFINDER does, and it works. Our claim is narrower than "RL beats search." We claim that a learned value of partial assemblies lets the search stay shallow almost everywhere, and that the horizon measurement tells us where depth is needed. If matched-budget beam search equals our tracer at every density level, we will report that the measured horizon simply justifies existing search designs. That is still the first quantitative account of why they work.
Why not just imitate the ground-truth oracle? Because the optimal action is known, DAgger-style imitation, which iteratively relabels the learner's own visited states with oracle actions, is a cheap and strong alternative to reward-driven learning. Imitation learns which action to take but not how much each alternative is worth. Without that value, there is no principled way to decide where to spend lookahead. We include DAgger with the same network as an additional baseline. If it matches our tracer on merges, we will conclude that the decisive ingredient is assembly-level input rather than value-based lookahead, and simplify accordingly.
Why not just add NEURD-style pruning rules after pairwise agglomeration? Rules repair merges after the fact, but NEURD cannot recover the splits that aggressive cutting creates. The continuous morphology result in An unsupervised map of excitatory neuron dendritic morphology predicts that fixed angle thresholds will remove real tortuous collaterals. We will run pairwise agglomeration followed by NEURD's heuristics on MICrONS and measure lost true path length alongside merges removed.
Proposed Experiments
The primary dataset is FlyTracing: 1.6 million pairwise connections over $3.2\times10^6\ \mu m^3$ of the adult fly brain, with FlyWire proofreading as ground truth. We split by brain region to prevent spatial leakage. For mammalian transfer we use MICrONS Minnie65 axons, and the IBEAM-mSEM cortex volume from PATHFINDER if its proofread reference is available.
We report four metrics:
- pairwise precision, recall, and F1;
- expected run length;
- merges and splits per millimetre of traced axon;
- plausibility evaluations per decision, as a measure of compute.
As an external check on merge judgments, we score our critic on ConnectomeBench's merge identification task. There the best reported result is about 62% for o4-mini (ConnectomeBench). A critic that does no better there has learned nothing about merges.
Three checks decide the outcome. The first is whether the horizon distribution is bimodal, with long horizons concentrated at merge-rejection decisions in dense neuropil. If nearly all decisions have $H=1$ under the pairwise-greedy base policy, the sequential framing is unnecessary, and we stop and publish that negative result. The second is whether the residual errors of the contrastive scorer fall disproportionately on decisions with $H\ge3$. This would explain why its 0.952 F1 coexists with stubborn merges. The third is whether the depth-adaptive tracer reduces merges per millimetre relative to both the pairwise scorer and matched-budget beam search, while full PPO adds nothing further. That result would confirm the effective-horizon prediction that shallow lookahead over a good value estimate is enough.
References
- Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing
- ConnectomeBench: Can LLMs proofread the connectome?
- Accelerating Neuron Reconstruction with PATHFINDER
- NEURD offers automated proofreading and feature extraction for connectomics
- An unsupervised map of excitatory neuron dendritic morphology in the mouse visual cortex
- The Effective Horizon Explains Deep RL Performance in Stochastic Environments
- Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning
- Recursive Reinforcement Learning