Depth Dependent Representational Stability in Neural Networks

40 concepts32 papers1 proposal

When a neural network learns a sequence of tasks, it often suffers from catastrophic forgetting—the tendency for new information to overwrite old knowledge. However, this forgetting is not distributed uniformly across the network's architecture. Instead, neural networks exhibit a hierarchical stability gradient, where the susceptibility to forgetting and representational drift is tied directly to a layer's depth.

The Anatomy of Non-Uniform Forgetting

In deep architectures, the "memory" of a network is spatially organized. Ramasesh et al. demonstrated that catastrophic forgetting is primarily concentrated in the output-proximal layers, while the input-proximal layers maintain remarkably stable representations across sequential tasks. This suggests that lower-level features—such as edges in images or basic syntax in text—are naturally generalizable and resilient to change.

This gradient of instability means that as information flows deeper into the network, it becomes increasingly fragile. Szatkowski et al. observed that the magnitude of representational change typically increases with layer depth, acting as a "medical check-up" for where knowledge is lost. While early layers act as a resilient memory of general features, the final layers undergo significant drift, erasing the specialized task logic required for previous objectives.

Why Deeper Layers Drift

The divergence in stability between shallow and deep layers stems from how they represent data. Mehrer et al. found that even when models share the same architecture and data, their deeper layers develop idiosyncratic "representational geometries" based on their initial random weights. In contrast, shallow layers converge to highly similar, consistent feature extractors regardless of initialization.

This idiosyncrasy makes deeper layers more prone to overfitting and task-specific erasure. Kim et al. provided a theoretical framework for this, proving that representational forgetting is positively correlated with layer depth. They argue that the increasing dimensionality of the feature space as information progresses through the hierarchy makes it easier for new tasks to displace old ones. Furthermore, Schneider notes that in the later stages of training, networks enter a "forgetting" phase where they discard input details to focus solely on classification—a process most aggressive in the final layers.

The Role of Architecture and Width

The relationship between depth and stability is not just a byproduct of training; it is influenced by the network's physical dimensions. Mirzadeh and Chaudhry showed that wider neural networks are naturally more resistant to forgetting because they operate in a "lazy training" regime with more orthogonal gradients. Conversely, increasing depth can actually destabilize the foundational layers by causing larger gradient updates to propagate back to the start of the network.

Interestingly, some architectures exhibit a "U-shaped" stability distribution. Nandan and Blondin discovered that in GPT-2, both the first and last layers remain remarkably stable, while the middle layers act as a more fluid space for internal computation. This suggests that the very end of the network, which prepares the final output, may converge on "universal" functional units that are as stable as the initial input-processing layers.

Engineering Stability for Continual Learning

Understanding this hierarchical gradient allows for more surgical interventions in continual learning. Rather than treating the network as a monolithic block, researchers have begun to apply stability constraints selectively:

  • Layer Freezing: Amrollahi et al. found that simply freezing early layers prevents the loss of foundational knowledge in clinical models.
  • Feature Replay: Instead of replaying raw data, Liu et al. proposed replaying high-level features to stabilize the deeper layers and the final classifier, where forgetting is most acute.
  • Dynamic Scaling: Pu et al. and Ramkumar both utilize strategies that maintain stable foundational representations in initial layers while allowing higher layers to remain plastic and adaptable for new task dynamics.
  • Hierarchical Supervision: Yan et al. treat the network as a stack of experts, using multi-level supervision to pass knowledge from stable shallow layers to the more volatile deep layers.

By aligning artificial architectures with the hierarchical stability observed in biological systems—where, as Curtis notes, different brain regions are assigned different levels of representational persistence—models can better balance the competing needs of plasticity for new learning and stability for memory retention.

Key Foundational Work

Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics (2021) — contributed 2 concept elements to this direction.
Generative Feature Replay For Class-Incremental Learning. (2020) — contributed 2 concept elements to this direction.
Probing Representation Forgetting in Continual Learning (2021) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_hierarchical_representation_stability_da0f32ec,
  title = {hierarchical representation stability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-representation-stability/rdr_da0f32ec4bd54d52ba5d9f46bf219a4c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_hierarchical_representation_stability_da0f32ec_v1,
  title = {hierarchical representation stability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-representation-stability/rdr_da0f32ec4bd54d52ba5d9f46bf219a4c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake