Introduction
A fundamental challenge in both artificial intelligence and neuroscience is the implementation of effective Working Memory (WM)—the ability to store and manipulate information over short intervals. Standard neural networks often struggle with tasks involving multiple timescales. For instance, an agent might need to remember a high-level "master rule" for an extended period while simultaneously needing to discard specific, transient details to avoid memory clutter and interference.
The original AuGMEnT (Attention-Gated MEmory Tagging) model was developed as a biologically plausible architecture, utilizing learning rules that mimic dopaminergic and attentional feedback in the brain. However, its "non-leaky" memory architecture forced it to retain all information until the end of a trial. In complex tasks with numerous distractor stimuli, this leads to significant memory interference, causing the network to fail at hierarchical benchmarks like the 12AX task. To address this, researchers have developed Hybrid AuGMEnT, which integrates memory units with varying decay rates to solve complex hierarchical tasks.
Methods: The Hybrid AuGMEnT Architecture
Hybrid AuGMEnT modifies the memory layer of the standard architecture to support multi-timescale dynamics through a dual-pathway system. The network features a "Regular" branch for standard feedforward processing and a "Memory" branch driven by "transient units" that respond to the onset (ON) or offset (OFF) of stimuli.
The core innovation lies in partitioning the memory units into two distinct groups:
- Non-leaky units: These maintain information indefinitely until a trial reset (decay constant $\phi = 1$), making them suitable for long-term context.
- Leaky units: These possess a fixed decay constant ($\phi < 1$), allowing them to "forget" information over time, which is ideal for filtering out transient distractors.
Crucially, the model maintains biological plausibility by employing a SARSA-based reinforcement learning scheme. It avoids non-biological methods like Backpropagation Through Time (BPTT) in favor of synaptic tags and eligibility traces—local chemical or physical markers at the synapse that "remember" which neurons contributed to an action until a global reward signal, such as dopamine, arrives.
Cognitive Tasks and Benchmarks
The model was evaluated on three cognitive tasks designed to test different memory demands:
- Saccade-AntiSaccade (S-AS): A validation task where the agent must look toward or away from a cue based on a previously established rule.
- Sequence Prediction: The agent predicts a final letter (Z or Y) based on an initial cue (A or X), with a variable number of meaningless distractor letters interspersed.
- 12AX Task: A famous hierarchical benchmark. The agent must identify "A-X" patterns if the most recent digit was "1", and "B-Y" patterns if the digit was "2". This requires storing the digit (outer loop) while processing and discarding various letter sequences (inner loop).
Key Results and Insights
The Hybrid AuGMEnT model demonstrated significant improvements in performance and robustness:
- 12AX Success: While the original AuGMEnT model had a 0% success rate on the 12AX task, the Hybrid version achieved a 100% success rate, typically converging within 34,000 trials.
- Robustness to Distractors: In sequence prediction tasks, the Hybrid model remained efficient even as the number of distractors increased. In contrast, a "Leaky Control" network (lacking non-leaky units) saw exponential increases in convergence time, highlighting the necessity of long-term maintenance units.
- Forgetting as a Functional Feature: In the 12AX task, the leaky units naturally "wash out" distractors, preventing them from interfering with high-level information. After training, the network automatically learns to use different units for different roles, specializing based on the required duration of information storage.
Discussion and Future Work
Unlike complex models like LSTMs or GRUs that use intricate "gates" to manage memory, Hybrid AuGMEnT achieves high performance using fixed decay constants and brain-like learning mechanisms. This provides a middle ground between high performance and biological plausibility.
However, several areas for future development remain. The decay rates ($\phi$) are currently fixed hyperparameters; future versions might allow the network to learn these timescales. Additionally, the model lacks an explicit "gate" to decide when to store information, a function typically associated with the basal ganglia. Finally, scaling the architecture to more complex, high-dimensional environments, such as Atari games, remains a significant future challenge.
Summary updated