Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks

Lukas Braun, Erin Grant, Andrew M Saxe

Overview

This paper analytically proves that a neural network's internal representations and its external behavior (function) are fundamentally dissociated: networks can implement the exact same function using wildly different internal codes. Crucially, the authors find that while generalization error and input noise do not force networks to develop "meaningful" representations, robustness to parameter noise (noise in the synaptic weights themselves) constrains networks to use task-specific representations that reflect the underlying structure of the data.

The Problem: Non-Identifiability in Neural Networks

In both neuroscience and artificial intelligence, researchers frequently compare "neural representations"—the patterns of activity in a hidden layer—to understand how a system operates. If two networks exhibit similar representations, it is often assumed they are performing the same computation.

However, neural networks are "non-identifiable." Because they are highly over-parameterized, there are infinitely many ways to configure the weights to achieve the exact same input-output mapping. This creates a fundamental gap in our understanding: we do not actually know how a network's internal code relates to its function. Can a network drift through entirely different internal states while its behavior remains perfectly stable? And if so many "messy" solutions exist, why do biological and artificial systems often converge on similar, structured representations?

Methods: Mapping the Solution Manifold

The authors tackle this problem using two-layer linear neural networks. These networks are mathematically tractable enough to allow for exact analytical proofs while still capturing the core dynamics of representation learning.

  1. Solution Manifold Partitioning: The authors define the "solution manifold"—the complete set of all possible weight configurations that solve a task perfectly. They then partition this manifold into subregions based on different constraints:
  2. GLS (General Linear Solution): Any configuration that solves the task.
  3. LSS (Least-Squares Solution): Solutions that minimize the norm of the overall network function.
  4. MRNS (Minimum Representation-Norm Solution): Solutions that minimize the activity of the hidden layer.
  5. MWNS (Minimum Weight-Norm Solution): Solutions that minimize the size of the weights.
A conceptual illustration of the solution manifold, where different weight configurations can yield the exact same network output.
  1. Random Walks: They implement random walks on these manifolds to observe how internal representations change while the output stays rigidly fixed.
  2. Nonlinear Validation: Finally, they use numerical simulations with ReLU networks on the MNIST dataset to confirm that their analytical findings from linear networks hold true in more complex, nonlinear settings.

Key Insights: Representational Drift and Parameter Noise

  • The "Elephant" in the Room: The authors demonstrate that for general solutions (GLS/LSS), representations are entirely "task-agnostic." You can mathematically force the hidden layer activity to form arbitrary shapes—even the shape of an elephant—while the network perfectly solves a hierarchical logic task. The internal code does not have to reflect the task structure at all.
  • Representational Drift is "Free": Because the solution manifold is high-dimensional, a network can undergo significant "representational drift"—changing its internal code completely—without any change in its outward behavior. This suggests that stable behavior in biological brains does not necessarily require stable neural representations.
A random walk on the solution manifold demonstrates that weights and hidden representations can change continuously while the network output remains perfectly stable.
  • Parameter Noise as a Sculptor: This is the paper's core revelation. Robustness to input noise (e.g., blurry images) does not force a network to develop structured representations. However, robustness to parameter noise (synaptic jitter) does. To remain robust to weight changes, the network must align its internal representations with the task's intrinsic structure, forcing it into task-specific representations.

Key Results

  • Dissociation Confirmed: Function and representation are formally dissociated. High representational similarity (measured via Representational Similarity Analysis or linear predictivity) does not guarantee functional alignment, and functional alignment does not require representational similarity.
  • Noise Sensitivity: In both linear and nonlinear networks, "expanded" or task-agnostic representations are significantly more sensitive to parameter noise than "minimal" task-specific ones.
  • Generalization is Insufficient: Surprisingly, achieving low generalization error (performing well on unseen data) does not, by itself, constrain a network to use minimal or task-specific representations.

Limitations and Future Directions

  • Linearity: While the authors show their results hold numerically for ReLU networks, the core analytical proofs are restricted to linear networks. Extending these exact proofs to deep, nonlinear architectures remains an open challenge.
  • Learning Dynamics: The paper focuses on the existence of solutions on the manifold, not how a specific learning algorithm (like Stochastic Gradient Descent) finds them. While they note that SGD has an "implicit bias" toward the structured solutions identified, fully characterizing this learning path is left for future work.
  • Biological Realism: The parameter noise model used is a simplified proxy for biological synaptic noise. Future work could explore more biologically detailed noise models to see how they shape neural coding.

Summary updated

Abstract

A foundational principle of connectionism is that perception, action, and cognition emerge from parallel computations among simple, interconnected units that generate and rely on neural representations. Accordingly, researchers employ multivariate pattern analysis to decode and compare the neural codes of artificial and biological networks, aiming to uncover their functions. However, there is limited analytical understanding of how a network’s representation and function relate, despite this being essential to any quantitative notion of underlying function or functional similarity. We address this question using analysable two-layer linear networks and numerical simulations in nonlinear networks. We find that function and representation are dissociated, allowing representational similarity without functional similarity and vice versa. Further, we show that neither robustness to input noise nor the level of generalisation error constrain representations to the task. In contrast, networks robust to parameter noise have limited representational flexibility and must employ task-specific representations. Our findings suggest that representational alignment reflects computational advantages beyond functional alignment alone, with significant implications for interpreting and comparing the representations of connectionist systems
BibTeX
@inproceedings{braun2025not,
  author = {Lukas Braun and Erin Grant and Andrew M Saxe},
  title = {{Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks}},
  booktitle = {International Conference on Machine Learning},
  year = {2025},
  url = {https://openreview.net/forum?id=YucuAuXMpT},
}
Figure