Iterative decoders for sparse error-correcting codes (LDPC codes, whose parity checks each touch only a few bits) usually fail for a local reason, not a global one. A handful of bits and checks form a small subgraph in which messages keep reinforcing a wrong answer. On erasure channels this subgraph is a stopping set: every check touching it touches it at least twice, so the decoder can never peel off a single unknown and stalls. On noisy channels the analogue is a trapping set or absorbing set. These small structures produce the error floor, the point at high signal-to-noise ratio where the error-rate curve stops falling. Classical code design trades the floor against the waterfall, the steep drop at moderate noise: changes that help one tend to hurt the other. Current neural decoders attack the floor with one monolithic correction network, or by swapping the whole decoder's weights at once.
We propose a decoder that treats each stalled subgraph as a "token" and sends it to one of several small specialist weight sets, called experts. Frames that converge never touch an expert, so waterfall behaviour is left exactly as the base decoder produces it. This design runs straight into expert collapse, the known failure of mixture-of-experts models in which a few experts capture all inputs and the rest never train. The standard remedy, forcing equal load on every expert, is the wrong target here. Floor failures come from a heavy-tailed mix of trapping-set types, and the frame error rate is set by whichever types remain uncorrected, not by how evenly work is spread. Our central proposal is to balance experts by residual failure mass: each expert's capacity is steered toward the trapping-set classes that still dominate the measured error rate. The waterfall–floor trade-off then reappears at a new level, as a question of how routing capacity is allocated, and we answer it there.
Core Claim
We make two claims, and the program can falsify each separately. The first is that the population of floor failures is heterogeneous enough that parameters tuned for one trapping-set class hurt another. If so, several cheap specialists applied only inside stalled regions beat a single post-decoder with the same parameter and iteration budget. The second is that when expert load is tied to residual failure mass rather than to uniform counts, the router neither collapses onto the common failure classes nor homogenizes the experts. The measurable result is a lower and flatter floor.
To make the second claim precise, let $c$ index trapping-set classes (for example $(a,b)$ types: $a$ bits whose neighbourhood leaves $b$ checks unsatisfied). Let $p_c$ be the probability that a failing frame contains class $c$ at the operating noise level, and let $r_c$ be the fraction of class-$c$ stalls the decoder still fails to resolve. The error floor is approximately
$$ \mathrm{FER}_{\text{floor}} \approx \sum_c p_c \, r_c , $$
This follows because floor failures are, to first order, disjoint local events. Uniform load balancing gives class $c$ capacity regardless of $p_c r_c$. Unconstrained routing gives it capacity roughly in proportion to $p_c$, which starves rare classes. We instead set per-expert capacity targets proportional to running estimates of $p_c r_c$, which puts capacity where the remaining error actually is.
Relation to Prior Work
The closest line of work is boosting for neural min-sum decoders by Kwak et al. In Boosting Learning for LDPC Codes to Improve the Error-Floor Performance and Boosted Neural Decoders: Achieving Extreme Reliability of LDPC Codes for 6G Networks, they harvest frames the base decoder fails on and train one post-decoder on them. They also share weights according to whether a check is satisfied or unsatisfied, which reduces memory. This work motivates ours and supplies our method. We reuse their failure harvesting, their unsatisfied-check weight sharing as the parameterization of each expert, and their importance-sampling data generation. We differ in replacing one post-decoder over whole frames with several experts applied only to stalled neighbourhoods.
Neural Window Decoder for SC-LDPC Codes keeps two global weight sets and swaps between them when a window fails. We ask whether routing local regions instead of the whole window avoids disturbing nodes that are already converging correctly. The static pre/post pipeline of Lowering the Error Floor of Error Correction Code Transformer depends on algebraic hard-decision decoders, so it does not carry over to LDPC codes. Learned Decimation for Neural Belief Propagation Decoders : Invited Paper breaks stalls by committing unreliable bits one at a time. It offers a complementary way to escape stalls, and we use it as a comparator on its short code.
On the coding-theory side, Stopping set analysis of repeat multiple-accumulate codes shows that raising minimum distance can shrink stopping distance. Improving the Thresholds of Generalized LDPC Codes with Convolutional Code Constraints shows that tuning for iterative threshold creates floors. Spatially coupled turbo-like codes: a new trade-off between waterfall and error floor escapes the trade-off asymptotically, but concedes that practical iterative decoders stay vulnerable to absorbing sets. These papers constrain our design by telling us the floor cannot be fixed by code construction alone at finite length. That makes the decoder the right place to intervene.
The mixture-of-experts literature supplies both the failure modes and the candidate fixes. The failure modes are starvation (Scaling Vision with Sparse Mixture of Experts) and homogenization under strong balancing losses (Advancing Expert Specialization for Better MoE, Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer). The fixes are exact-capacity assignment in BASE Layers: Simplifying Training of Large, Sparse Models and Expert Choice in Routers in Vision Mixture of Experts: An Empirical Study. Every one of these was studied with billions of roughly balanced tokens, which is the opposite of our regime of rare, heavy-tailed failure events. Spatial Mixture-of-Experts routes individual grid locations and trains its router from backpropagated error. We borrow that routing-classification idea. Its authors leave irregular graphs such as Tanner graphs open, which is exactly the setting we address.
Research Program
The decoder is a neural min-sum decoder with shared default weights. A stall detector runs every iteration. It flags any check that has stayed unsatisfied for $k$ consecutive iterations, and grows the flagged checks into connected regions: the flagged checks plus their variable nodes within two hops. Each region is summarized by a small feature vector: number of unsatisfied checks, number of variables, degree profile, count of message-sign flips over recent iterations, and mean reliability magnitude. A router maps this vector to one of $E$ experts or to a "no expert" option. Each expert is a set of unsatisfied-check-style edge weights and a damping factor applied only to edges inside the region for the next $T$ iterations. An expert therefore adds tens of parameters, not a network. Routing touches only the rare stalled regions, which limits the irregular-memory cost that Spatial Mixture-of-Experts observed.
Training has three stages. First, failure frames are harvested from the base decoder with the importance-sampling pipeline of Kwak et al. Second, each stalled region is labelled offline with its trapping-set class wherever enumeration is feasible, and the label distribution gives initial estimates of $p_c$. Third, router and experts are trained jointly. Experts pick regions under capacity targets proportional to the running estimate of $p_c r_c$. That estimate is updated each epoch from the decoder's own failures on a held-out harvested pool. Class labels enter the router only as an auxiliary signal that decays during training. The router must eventually work from local features alone, because at deployment the class is unknown and on long codes it may not be enumerable. Throughout training we log expert load, the entropy of the router's assignments, and the mutual information between expert identity and trapping-set class. These logs let us see collapse or homogenization directly rather than infer it from error rates.
Baselines
The naive reference is weighted min-sum with the same maximum iteration count. It shows the untreated floor that every method must improve on.
The strongest existing method is the boosted neural min-sum decoder with unsatisfied-check weight sharing from Boosting Learning for LDPC Codes to Improve the Error-Floor Performance. It reports that frame error rate at floor onset "dropped by over two orders of magnitude" relative to weighted min-sum on WiMAX, WiFi and 5G NR codes, using "only one-third of the iterations (50 vs. 150)" of classical post-processing. The 6G follow-up, Boosted Neural Decoders: Achieving Extreme Reliability of LDPC Codes for 6G Networks, reports "FER below $10^{-9}$" at 5 dB on a 5G code and a "$10^4$ (10,000x) improvement in FER" over weighted min-sum. We retrain it ourselves on the identical harvested failure pool, with the same total iteration budget and a tuning budget matched to ours in trials. We also give it the same total parameter count as base weights plus all experts, by widening its weight sharing. Beating it means a lower frame error rate at matched iterations and parameters at every signal-to-noise point in its floor region, or equal error rate with fewer average iterations. In either case the waterfall region must be unchanged within Monte Carlo confidence.
On the spatially coupled (3,6) code with $N=20{,}000$, the comparator is the neural window decoder with global weight swapping from Neural Window Decoder for SC-LDPC Codes. It reports a "3x reduction in Frame Error Rate (FER)" at 2.5 dB over the baseline window decoder. Beating it means a lower rate of error propagation along the chain at equal window size. On the (128,64) CCSDS code, learned decimation from Learned Decimation for Neural Belief Propagation Decoders : Invited Paper reports reaching "within 1 dB of maximum-likelihood decoding" at block error rate $10^{-4}$. We compare at equal decoding latency.
Four ablations isolate the key ingredients. The first replaces failure-mass capacity with uniform Expert Choice capacity. The second uses a standard auxiliary load-balancing loss with token choice. The third routes whole frames rather than stalled regions. The fourth is an oracle that dispatches each region to a post-decoder trained only on its true enumerated class, which bounds what class-based specialization can deliver.
Why not just…?
Why not just widen the single post-decoder? At the parameter scale involved, this is the most direct test of whether specialization matters, and the parameter-matched boosted baseline runs exactly that test. If it matches our decoder, the heterogeneity claim is false and we will report that. Two things suggest it might not match. Kwak et al. reached their results by sharing weights more coarsely, not more finely, which hints that the gains come from targeting failures rather than from raw capacity. And per-class residual rates will show whether one dominant class absorbs the monolithic decoder's fit.
Why not just classify the trapping set and look up a dedicated decoder? This is the real threat, which is why it appears in the baselines as the oracle. On short codes whose trapping sets can be enumerated, it may well match the learned router. Our argument for learning is confined to two cases. One is long 5G codes, where enumeration is incomplete. The other is ambiguous syndromes, where several classes produce the same pattern of unsatisfied checks. We measure the router's advantage on classes held out of the enumeration. If there is no advantage there, the right conclusion is that routing should be done by lookup, and the collapse and balancing results still apply to that lookup's capacity allocation.
Why not just add more boosting stages? A cascade of post-decoders, each trained on the previous stage's failures, reshapes the failure distribution without any router. We run a two- and three-stage cascade at matched iterations. A cascade applies each stage to the whole frame in sequence, so its cost in added iterations grows with depth. Local experts act in parallel inside separate regions of the same frame, and an iteration-matched comparison settles which is better.
Why not just use standard load-balancing? Evidence from language models shows that strong balancing homogenizes experts (Advancing Expert Specialization for Better MoE). Our ablation measures this directly through the mutual information between expert identity and trapping-set class.
Proposed Experiments
We evaluate on the WiMAX, WiFi and 5G NR LDPC codes used by Kwak et al., the (128,64) CCSDS code, and the protograph (3,6) spatially coupled code, all on the additive white Gaussian noise channel. Waterfall points use direct Monte Carlo simulation. Floor points below roughly $10^{-7}$ use importance sampling biased toward harvested trapping sets, validated by direct simulation where the overlap is feasible. We also test sensitivity to the sampling bias, because Kwak et al. flag distribution shift from importance sampling as a known risk. For every method we report frame error rate against signal-to-noise ratio, average and worst-case iteration counts, parameter memory, and residual failure rate per trapping-set class.
The experiments are decisive in four ways. First, if the parameter-matched monolithic post-decoder equals our decoder, the specialization claim fails. Second, failure-mass balancing must outperform uniform Expert Choice specifically on rare classes, while mutual information between expert and class stays high. If both balancing rules give the same per-class residuals, the collapse framing adds nothing. Third, the waterfall curve must be statistically unchanged from the base decoder, which confirms that routing only stalled regions protects the waterfall. Fourth, when the router is trained at one signal-to-noise ratio and tested at others, where class frequencies shift, the adaptive capacity targets should track the new failure mix. Fixed targets should not.
References
- Boosting Learning for LDPC Codes to Improve the Error-Floor Performance
- Boosted Neural Decoders: Achieving Extreme Reliability of LDPC Codes for 6G Networks
- Neural Window Decoder for SC-LDPC Codes
- Lowering the Error Floor of Error Correction Code Transformer
- Learned Decimation for Neural Belief Propagation Decoders : Invited Paper
- Stopping set analysis of repeat multiple-accumulate codes
- Improving the Thresholds of Generalized LDPC Codes With Convolutional Code Constraints
- Spatially coupled turbo-like codes: a new trade-off between waterfall and error floor
- Scaling Vision with Sparse Mixture of Experts
- Advancing Expert Specialization for Better MoE
- Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer
- BASE Layers: Simplifying Training of Large, Sparse Models
- Routers in Vision Mixture of Experts: An Empirical Study
- Spatial Mixture-of-Experts