Autonomous Continual Learning and Knowledge Transfer via Cortex-Inspired Neural Assemblies

Kevin L. McKeearXiv 2026 · arXiv

TL;DR

Functional Task Networks (FTNs) introduce a brain-inspired method for artificial neural networks to learn new tasks sequentially without forgetting old ones. By using a sparse, optimization-based router to select "neural assemblies" from a large bank of features, FTNs autonomously manage parameter isolation and knowledge transfer without requiring manual task labels.

What Problem Does This Paper Address?

Artificial Neural Networks (ANNs) are notoriously bad at "continual learning." When you train them on a new task, they often overwrite the weights used for previous tasks—a phenomenon known as catastrophic forgetting (CF).

The paper highlights two specific challenges:

  1. Concept Shift: The underlying rules or labels change (e.g., the same input now means something else).
  2. Domain Shift: The input data distribution changes (e.g., moving from daylight to nighttime images).

Most current solutions require a "task oracle" (someone telling the network exactly when a task has changed) or struggle to reuse old knowledge for new problems. This paper aims for an autonomous system that can handle both retention and transfer in a "block-sequential" regime, where the network sees one task for a long time before moving to the next.

Figure 1: Catastrophic forgetting under block-sequential nonlinear regression. Red curves show predictions after each block. Fitting a new input region degrades earlier regions; fitting a new target function overwrites the previous function.

Methods

The authors propose Functional Task Networks (FTNs), which are modeled after the mammalian neocortex. The core idea is to treat a network not as a single monolithic block, but as a distributed mixture of experts.

  • Bank of Experts: The network consists of a large bank of small, trainable units (like MLPs or Low-Rank Adapters).
  • Optimization-Based Routing: Instead of a fixed router, FTN uses a small "support set" of labeled examples to optimize a routing mask. This mask selects the best $k$ experts for the current task.
  • Sparse, Binary Masks: By using a top-$k$ selection, the network creates a sub-assembly. Units not selected are frozen, protecting them from being overwritten.
  • Mask-Overlap-Filtered Replay: To handle shared components between related tasks, the system uses a replay buffer. It specifically samples old data that overlaps with the current task's active units, mitigating interference where it's most likely to happen.
Two possible architectures that implement continual learning via FTN. A large bank of processing units is discretely and sparsely gated into a combination function.

Datasets

The researchers evaluated FTN on several benchmarks designed to test different types of shifts:

  • Synthetic Task Family: 12 binary classification tasks with controlled overlap (correlated vs. orthogonal) to test how well the network identifies shared concepts.
  • Shared-Shuffle MNIST: A variant of MNIST where digit-to-label mappings are permuted, testing concept shift.
  • Permuted & Rotated MNIST: Standard benchmarks for domain shift where pixels are scrambled or images are rotated.

Key Insights and Ideas

  • Routing as Inference: The most novel "aha moment" is treating routing as an optimization problem at inference time. By finding the mask that minimizes loss on a small support set, the network can autonomously "recall" an old task or "assemble" a solution for a new one.
  • Distributed Assemblies: Knowledge isn't stored in a single "expert" but in a distributed population. This redundancy allows for forward transfer (a "head-start" on new tasks) because a new task can recruit partially trained units that already know something useful.
  • Loss Ordering: The authors show that a mixture of partially trained and random features often performs better on a new, related task than a purely random initialization, providing a natural mechanism for knowledge transfer.
Figure 2: Schematic loss ordering of candidate neural assemblies. Lower po- sitions indicate lower loss on the target task. Relative performance depends on task structure and feature usefulness.

Key Results

  • Retention: FTN retains solutions nearly as well as "oracle" methods that are manually told when tasks change. On synthetic tasks, it reached a forgetting measure (FM) of just 0.029, compared to 0.396 for naive fine-tuning.
  • Forward Transfer: FTN achieved the lowest "head-start loss" among all matched-architecture comparators. This means that before any training on a new task, the optimized mask already provides a significantly better starting point.
  • Geometry Tracking: The learned overlap between masks accurately reflected the ground-truth relationship between tasks ($r_{share} \approx 0.62$), proving the network was actually learning the underlying structure of the problem space.

Limitations and Future Work

  • Computational Overhead: The primary limitation is the extra computation needed to optimize the router mask between gradient steps. For large models (like the LoRA implementation), this requires evaluating the entire model multiple times per step.
  • Support Set Requirement: The system relies on having a small labeled support set available to configure the mask.
  • Future Directions: The authors suggest moving toward approximate Bayesian updating of the masks to make the process more scalable and faster, potentially removing the need for iterative optimization during inference.

Summary updated

Abstract

Unlike the brain, artificial neural networks (ANNs) suffer from "catastrophic forgetting": complete overwriting of solutions when re-training on new functions ("concept shift") or on a disjoint domain of the current function ("domain shift"). ANNs require each batch of data to be roughly representative of the complete distribution, a typically expensive and impractical requirement. In this paper, we introduce Functional Task Networks (FTNs), a method of continual learning and knowledge transfer modeled after several properties of the mammalian neocortex. FTN is formulated as a distributed, stochastic mixture of experts in which a sparse, binary routing mask is computed at inference time using a small support sample of input-output pairs. Per-batch reconfiguration of the router during training, together with mask-overlap-filtered replay, shapes a basis of overlapping, partially trained subnetworks. Unrelated tasks route through mostly separate assemblies, related tasks overlap on shared components. On block-sequentially trained continual learning and knowledge transfer tasks FTN retains solutions nearly as well as disjoint-mask isolation and attains the lowest pre-update head-start loss among matched-architecture comparators. The learned mask-overlap geometry tracks the known concept-sharing structure where available. We show experimentally that under correlated concept shift, support-set mask reconfiguration is necessary even for retention. Under domain shift, the distributed bank with replay is sufficient to handle retention without the configurer. Forward transfer, or head-start, and backward transfer, are also contributed by support-set mask configuration.
BibTeX
@misc{mckee2026autonomous,
  author = {Kevin L. McKee},
  title = {{Autonomous Continual Learning and Knowledge Transfer via Cortex-Inspired Neural Assemblies}},
  year = {2026},
  eprint = {2611.11717},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.11717},
}
Figure