Abstract
Working memory (WM) is a fundamental component of intelligence, yet most computational models rely on simplified, low-dimensional inputs. This study investigates how recurrent neural networks (RNNs) represent and maintain high-dimensional, naturalistic 3D objects across various N-back tasks. We demonstrate that RNNs do not utilize static "slots" for memory but instead employ dynamic, chronological subspaces that evolve over time. Our analysis reveals that gated architectures (GRUs/LSTMs) favor task-specific representations, while vanilla RNNs utilize shared subspaces. Furthermore, we observe a trend toward feature compression within the latent space, suggesting an efficient dimensionality reduction strategy for working memory maintenance.
Introduction
The ability to hold and manipulate information over short durations is essential for complex cognition. Traditional models of working memory often simplify this process by using "one-hot" categorical inputs. However, real-world stimuli are multidimensional; for instance, an object possesses a category, a specific identity, and a spatial location simultaneously. This research addresses the gap in our understanding of how neural networks maintain such high-dimensional information across diverse cognitive demands, specifically examining the internal geometric structure of these representations.
Methods
The researchers implemented a two-stage "sensory-cognitive" model to simulate the processing of naturalistic stimuli. The first stage, Perception, utilizes a pre-trained ResNet50 convolutional neural network (CNN) to generate high-dimensional visual embeddings from 3D object images. The second stage, Cognition, feeds these embeddings into an RNN (testing vanilla RNN, GRU, and LSTM architectures) tasked with performing various memory-intensive operations.
The models were trained on nine variants of the N-back task (1-back, 2-back, and 3-back) across three feature dimensions: Location, Identity, and Category. Stimuli were sourced from the ShapeNet dataset, featuring four object categories with unique identities rendered at multiple locations and viewing angles. To analyze the internal representations, the authors employed decoder-based analyses and Orthogonal Procrustes analysis to track how neural activations transform from initial encoding to long-term maintenance.
Key Insights and Results
All models achieved high performance, with accuracy exceeding 95% on training sets and 90% on validation sets involving novel viewing angles. A key finding was the "Pack-Rat" strategy: multi-task RNNs simultaneously encode both task-relevant and task-irrelevant information, maintaining a full representation of the object regardless of the specific task demand.
Geometric analysis revealed a fundamental difference between architectures. Vanilla RNNs tend to use shared subspaces across different tasks, whereas gated RNNs (GRUs and LSTMs) develop highly task-specific "private" subspaces. This specialization in gated units suggests a trade-off: while they are more efficient at specific tasks, they may struggle with zero-shot generalization to new task demands.
Furthermore, the study challenged the traditional "slot-based" model of working memory. Instead of fixed containers, RNNs utilize relative chronological subspaces. Information is organized based on its "age" (the number of timesteps since it was seen), and representations are dynamically rotated into new age-specific subspaces as time progresses. Interestingly, the RNNs also exhibited feature compression (de-orthogonalization), where object features became more overlapping compared to the initial CNN perceptual space, likely serving as a form of dimensionality reduction for the working memory bottleneck.
Discussion
While the models demonstrated robust performance and view-invariance, a significant "generalization gap" remains; performance dropped when encountering entirely novel object identities. This suggests that the cognitive components of these models are still partially tied to the specific objects encountered during training.
Future work should explore whether these chronological subspaces persist in other working memory paradigms and how these findings scale with larger architectures like Transformers. The discovery of chronological subspaces provides concrete, testable predictions for neuroscientists investigating biological working memory in the brain.
Summary updated