Do Mice Grok? Glimpses of Hidden Progress During Overtraining in Sensory Cortex

Tanishq Kumar, Blake Bordelon, Cengiz Pehlevan, Venkatesh N. Murthy, Samuel J. GershmanarXiv 2024 · DOI · arXiv

Introduction

In both artificial intelligence and cognitive psychology, "overtraining"—the continuation of practice long after a task has been mastered—is known to enhance generalization and robustness. In machine learning, this phenomenon is often linked to "grokking," where a model suddenly achieves high test accuracy long after training loss has seemingly plateaued. However, the neural underpinnings of this process in the biological brain remain poorly understood. This study investigates whether the sensory cortex continues to refine its internal representations during behavioral plateaus and explores the mathematical principles, such as margin maximization, that might drive these "hidden" changes.

Methods

The researchers employed a dual approach combining neural data reanalysis and computational modeling. They re-examined a dataset of tetrode recordings from the posterior piriform cortex (PPC) of four mice. These animals were trained to discriminate a specific target odor from a diverse set of non-target mixtures, continuing for 18 days after reaching behavioral mastery.

To interpret these findings, the authors developed two models: a simple multi-layer perceptron (MLP) and a biologically plausible recurrent model with sparse coding. They analyzed neural state space using Representational Similarity Matrices (RSM), Linear Discriminant Analysis (LDA) for decoding, and Support Vector Machine (SVM) margins to quantify the robustness of decision boundaries.

Results: Behavioral Plateau vs. Neural Progress

A central finding is the decoupling of behavioral performance and neural representation. While the mice maintained a stable, high-accuracy response rate (licking behavior) from the eighth day of training onwards, the underlying neural activity in the PPC continued to evolve.

Behavioral performance vs. neural decoding accuracy during overtraining.

As shown in the figure above, the decoding accuracy of odor identity from neural populations increased steadily throughout the overtraining period. This suggests that the brain is not merely maintaining a learned state but is actively refining the separability of neural representations even when the animal's choice behavior remains at a ceiling.

Implicit Margin Maximization

The authors propose that the brain acts as a max-margin classifier. During overtraining, rather than just maintaining "correctness," the neural circuitry appears to maximize the distance between the decision boundary and the most ambiguous or "hard" examples.

Normalized margins for hard examples increase over time.

This "margin growth" provides a geometric signature of hidden progress. In computational simulations, the authors found that non-vanishing loss functions, such as Cross-Entropy, are essential for this effect. Unlike Hinge loss, which stops providing gradients once a sufficient margin is reached, Cross-Entropy continues to push representations apart, leading to the "grokking" of difficult, unseen probe odors.

Discussion and Implications

These findings offer a potential explanation for the "overtraining reversal effect" in psychology, where overtrained animals adapt more quickly to task reversals. By building more robust and "rich" features during the plateau phase, the brain creates a representational structure that is more flexible and easier to repurpose for new contingencies.

While the study is limited by its observational nature and small sample size, it provides a compelling bridge between deep learning theory and systems neuroscience. Future work should investigate the specific biological mechanisms—perhaps related to metabolic efficiency or unsupervised homeostatic plasticity—that prevent the learning signal from vanishing once behavioral goals are met.

Summary updated

Abstract

Does learning of task-relevant representations stop when behavior stops changing? Motivated by recent theoretical advances in machine learning and the intuitive observation that human experts continue to learn from practice even after mastery, we hypothesize that task-specific representation learning can continue, even when behavior plateaus. In a novel reanalysis of recently published neural data, we find evidence for such learning in posterior piriform cortex of mice following continued training on a task, long after behavior saturates at near-ceiling performance ("overtraining"). This learning is marked by an increase in decoding accuracy from piriform neural populations and improved performance on held-out generalization tests. We demonstrate that class representations in cortex continue to separate during overtraining, so that examples that were incorrectly classified at the beginning of overtraining can abruptly be correctly classified later on, despite no changes in behavior during that time. We hypothesize this hidden yet rich learning takes the form of approximate margin maximization; we validate this and other predictions in the neural data, as well as build and interpret a simple synthetic model that recapitulates these phenomena. We conclude by showing how this model of late-time feature learning implies an explanation for the empirical puzzle of overtraining reversal in animal learning, where task-specific representations are more robust to particular task changes because the learned features can be reused.
BibTeX
@misc{kumar2024do,
  author = {Tanishq Kumar and Blake Bordelon and Cengiz Pehlevan and Venkatesh N. Murthy and Samuel J. Gershman},
  title = {{Do Mice Grok? Glimpses of Hidden Progress During Overtraining in Sensory Cortex}},
  year = {2024},
  eprint = {2411.03541},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.03541},
}
Figure