Pathway Micro-Modules for Learning New Genes Without Forgetting Old Ones

Single-cell RNA sequencing measures how strongly each of roughly twenty thousand genes is expressed in each individual cell. Models trained on this data increasingly need to grow: new gene panels, new assays and new studies add genes that the original model never saw. Retraining from scratch every time is wasteful. Adding genes to an existing model makes it forget what it learned about the old genes, and Gene Incremental Learning for Single-Cell Transcriptomics measured this forgetting directly. That study also found that the standard fix borrowed from computer vision, distillation (forcing the updated model to mimic the old model's outputs), makes cell classification worse rather than better. The only fix that worked well was replay, which means storing about a thousand raw cell profiles and retraining on them.

We propose borrowing a structural idea from education research on micro-learning. That literature breaks a dense curriculum into short, self-contained units, keeps a set of prerequisites that every unit can rely on, and then adds an explicit step that knits the units back together, because isolated units otherwise leave knowledge in silos (Application of Flipped Classroom Based on MOOCs and Micro lesson in the Teaching Process of Pathology). Applied to a single-cell model, each new batch of genes becomes a small network that reads only one biological pathway's genes, plus a shared panel of "base" genes. Once trained, that small network is frozen. A lightweight integration layer combines all of them, and it is refreshed at each stage by replaying compact pathway summaries of past cells instead of raw expression profiles. The claim to test is specific. Because a pathway summary is a few numbers per pathway rather than thousands of gene counts, this design should be able to remember many more cells within the same memory budget. If so, it should beat raw replay at equal memory while being interpretable by construction.

Core Claim

Forgetting in gene-incremental learning has two separable sources. The first is drift inside the parts of the model that encode old genes. The second is drift in the shared classifier that reads the cell representation. Freezing per-pathway modules removes the first source completely. The second source is what actually drives the classification losses reported for the baseline (1.816% average accuracy drop) and for distillation (2.473% drop) in Gene Incremental Learning.

We claim the second source can be controlled cheaply. Each frozen module emits a low-dimensional activity vector for its pathway, using the 4-dimensional per-pathway latents of pmVAE. We therefore store those vectors for past cells instead of the cells themselves. Under a fixed byte budget this "latent pathway replay" holds one to two orders of magnitude more cells than raw replay. It should match or beat raw replay's forgetting, and the gain should come specifically from pathway-aligned grouping rather than from parameter isolation alone. That second part is the non-obvious, falsifiable piece. It is also the reason the idea is not simply "freeze old parameters," which would trivially prevent internal drift for any grouping.

Research Program

The model has three components.

The first is a base encoder over the retained base-gene panel. We keep the Base Gene Mechanism from Gene Incremental Learning, which exists precisely so that each increment still sees biologically meaningful partial cells.

The second is a growing set of pathway micro-modules. Each one is a small adapted MLP in the style of A Simple and Comprehensive Benchmark for Single-Cell Transcriptomics, which showed a 0.64M-parameter adapted MLP reaching 81.4% accuracy, close to a 21.87M-parameter Transformer. Each module takes base genes plus the genes of one Reactome pathway. It is trained with pmVAE's combination of a local reconstruction loss (rebuild this pathway's own genes) and a global reconstruction loss (rebuild the whole profile), and it is frozen after its stage.

The third is an integration layer, a small attention-free mixer over module outputs that feeds the cell-type classifier and the masked-expression regression head.

Genes not covered by any curated pathway are a known weakness of pathway models; both pmVAE and scPML effectively discard unannotated genes. We handle them by forming residual modules from co-expression clusters computed on the increment's own data.

At each new stage, training proceeds in a fixed order. New modules are trained first. The integration layer is then refit on the current stage's cells together with a buffer of stored module-output vectors from earlier stages. The analysis also asks a question taken directly from the micro-learning lens: does the order in which pathways arrive matter? We compare prerequisite-first ordering, which follows the Reactome hierarchy from general to specific pathways, against reversed and random orders. This tests whether "units with prerequisites" is a useful idea for models or only a metaphor.

Baselines

The naive reference point is the GIL baseline: train on the new genes in each increment with no protection. The file reports a regression-loss forgetting of 0.279 and a 1.816% average classification drop for it in the two-stage setting (Gene Incremental Learning).

The strongest existing method is Gene Replay with a 1,000-cell buffer, which the same paper reports cuts regression forgetting to 0.037 and the classification drop to 1.075%. We will run it ourselves using the same backbone family, the same increments and the same tuning budget. We will compare against it at two settings: its original 1,000-cell size, and an equal-bytes setting where its buffer is sized to match our latent buffer's memory. Beating it means a lower classification drop than raw replay at equal memory, with regression forgetting no worse than 0.037 on the two-stage protocol. The gap must be larger than the seed-to-seed spread across at least five seeds.

Gene Distillation (0.212 regression drop, 2.473% classification drop) is included to confirm we reproduce the reported failure. Joint training on all genes at once serves as the upper bound.

The ablation that isolates our key ingredient is the same frozen-module architecture with genes grouped at random into modules matched to the pathway modules' size distribution. It uses identical latent replay. If random grouping performs as well as pathway grouping, the benefit is parameter isolation, not biology, and we will report it that way. A second ablation removes the local reconstruction loss, to test whether pmVAE's local-global balance is what keeps module outputs informative enough to replay.

Why not just…?

Why not just use a bigger raw replay buffer? This is the real threat. Replay already works well on this problem, and if memory is cheap, a larger buffer may close most of the remaining gap. Our answer is not that replay fails. We claim that at any fixed memory budget, compact pathway summaries buy more remembered cells. The equal-bytes comparison settles this directly. We will also trace a full memory-versus-forgetting curve for both methods, so a reader can see where, if anywhere, raw replay catches up. Storing summaries instead of raw profiles has a secondary advantage when raw patient data cannot be retained, but we treat that as a side benefit, not the justification.

Why not just retrain from scratch whenever genes are added? When all historical data is still available and compute is cheap, joint retraining is the right answer, and it is our upper bound. The incremental setting matters when earlier data cannot be revisited or when models are updated often. The question is how close we get to joint training at a fraction of its per-stage compute. We will report per-stage training time and the remaining accuracy gap to joint training.

Why not just freeze old parameters with any grouping, as in column-growing continual learning? This is exactly the random-grouping ablation. Freezing alone guarantees no drift inside modules. It does not guarantee that the small summaries are informative enough for the integration layer to recover old decision boundaries. We expect pathway grouping to help because coordinated pathway activity is more stable across cells and noise than individual genes, as scPML argued for cross-platform robustness (0.87 versus Seurat's 0.81 mean accuracy). If the ablation shows no difference, the pathway story is wrong and we will say so.

Why not just use a large pretrained Transformer foundation model? Prior evidence suggests scale is not the bottleneck. The benchmark above found a 0.64M adapted MLP within a few points of a 21.87M Transformer, and SC-Arena found frontier language models failing to reach passing thresholds on single-cell tasks. A Transformer would also still need its own anti-forgetting mechanism when the gene vocabulary grows. We include an adapted-MLP monolith trained on the same increments for reference, but not a large Transformer, for compute reasons.

Proposed Experiments

The primary benchmark is the GIL protocol on the IColitis and Myeloid datasets, measured by gene-wise regression forgetting and gene-based classification accuracy drop exactly as defined in Gene Incremental Learning. We first reproduce the two-stage numbers. We then extend to four- and six-stage curricula, which that paper did not explore and where differences between replay variants should widen.

The decisive check is a single plot per dataset. It shows classification drop against buffer memory in bytes for raw replay, latent pathway replay, and latent random-group replay. The proposal succeeds only if the pathway curve lies below the raw replay curve and below the random-group curve over a meaningful range of budgets.

The ordering experiment compares prerequisite-first, reversed and random pathway orders on the six-stage protocol.

For interpretability, we test whether modules learned early in the stream still identify the right biology after later stages. We train a module stream that includes the interferon pathway on the pmVAE Kang et al. interferon-beta PBMC data, then ask whether the frozen interferon module still separates stimulated cells after later stages. pmVAE reports 0.97 accuracy for its interferon-alpha/beta module trained all at once, and we use that as the reference. The pathway hypergeometric test from the same work checks that the top-ranked modules remain the biologically expected ones.

Robustness to batch effects is a secondary check. We run cross-platform PBMC annotation with scPML's protocol (scPML reports 0.87 mean accuracy), with each platform arriving as a new stage. No reported result was found on Lacuna for incremental versions of this protocol, so the comparison there is against our own joint-training run.