TL;DR
Neural networks are more than just the functions they compute; they are historical records of their own training. This paper demonstrates that a network's final weights contain decodable "fingerprints" of its initial random state and the specific sequence of tasks it encountered, proving that learning trajectories leave persistent, identifiable traces even after extensive training.
What Problem Does This Paper Address?
When we train multiple neural networks on the same task, they all end up performing well, but their internal weight configurations are never identical. For a long time, researchers have wondered: are these differences just random noise—alternative but equivalent solutions—or do they encode information about how the network got there?
The authors investigate whether a trained network "remembers" its past. Specifically, they ask if the final state of a network can reveal its initial starting weights, the order in which it learned different tasks, and whether a network bumped off its training path will ever return to its original trajectory.
Methods
The researchers used a combination of weight-space analysis and linear probing:
- Symmetry Alignment: To compare different networks fairly, they used "Git Re-Basin" techniques (specifically the Hungarian algorithm) to account for neuron permutations and scaling. This ensures that two networks aren't seen as "different" just because their internal neurons are in a different order.
- Identity Retrieval: They tested if a trained network could be matched back to its specific random initialization among a set of candidates.
- History Probing: They trained networks on sequences of tasks (e.g., Task A then B, or B then A) followed by a common final task (Task C). They then used linear decoders to see if the final weights still held the "memory" of that early task order.
- Perturbation Experiments: They intentionally added noise to weights during training to observe if the "disturbed" network would converge back to the original unperturbed path.
Datasets
The study focused on three standard image classification benchmarks:
- MNIST: Handwritten digits.
- Fashion-MNIST: Clothing items.
- Kuzushiji-MNIST (KMNIST): Classical Japanese characters.
Key Insights and Ideas
- The Persistent Fingerprint: Training does not erase a network's origins. Even though the overall geometry of a group of networks is reorganized during training, each individual network retains a unique signature of its initialization.
- Weights are Better History Books than Activations: While internal representations (measured via CKA) show some historical influence, the raw weights are far more reliable for decoding a network's past.
- Functional Recovery $\neq$ Geometric Convergence: If you perturb a network's weights, it will quickly recover its performance (accuracy), but it will not return to its original path in weight space. It finds a new, equally good solution instead.
- The "First Impressions" Effect: In multi-task learning, the earliest tasks encountered by a network leave the most decodable traces in the final weights, suggesting that early training stages have a foundational impact on the final model structure.
Key Results
- 100% Identity Retrieval: In every test case across all three datasets, the researchers were able to correctly match a fully trained network back to its specific initialization among ten candidates.
- Perfect History Decoding: A simple linear probe could determine the order of two early training tasks with 100% accuracy on 80 held-out networks, even after those networks had spent ten times as much time training on a completely different final task.
- Low-Loss Connections: The authors found that 2,157 out of 2,160 pairs of "perturbed" and "unperturbed" solutions were connected by a straight line of low loss. This supports the algorithmic-attractor hypothesis: networks are drawn to a broad region of successful solutions rather than a single point, allowing them to stay distinct while performing the same function.
Limitations and Future Work
- Model Scale: The study used relatively small, single-hidden-layer fully connected networks. Whether these "historical traces" remain as decodable in massive architectures like Transformers or deep ResNets remains an open question.
- Task Complexity: The experiments were limited to image classification. More complex curricula, such as reinforcement learning or language modeling, might show different patterns of historical persistence.
- Recognition vs. Reconstruction: While the study shows we can recognize an initialization from a list of candidates, we cannot yet reconstruct the exact initial state from the final weights. Future work could explore the limits of how much of the full trajectory can be mathematically recovered.
Summary updated