TL;DR
When faced with ambiguous data, different neural architectures (LSTMs, CNNs, and Transformers) exhibit strikingly different "hidden" preferences, or inductive biases. By measuring "description length"—how easily a model can represent a rule—researchers found that LSTMs can learn to count or add from a single example, while Transformers and LSTMs inherently prefer hierarchical rules over linear ones, unlike CNNs.
The Problem: Poverty of the Stimulus
In machine learning, we often encounter the "poverty of the stimulus": training data is too sparse to point to a single correct way to generalize. For example, if a model learns that $f(3) = 6$, should it assume $f(4)$ is $7$ (add 3), $8$ (multiply by 2), or $6$ (a constant function)?
While humans have strong innate biases—like assuming language has a hierarchical structure—we don't fully understand the "gut instincts" of our sequence-to-sequence (seq2seq) models. This paper maps out these inductive biases to understand why certain architectures succeed or fail at specific tasks, particularly in low-data regimes.
Methods for Measuring Bias
The authors propose a formal framework to "interrogate" models by training them on highly ambiguous, minimal datasets (often just one example) and observing which rule they follow on new inputs.
They introduce two primary ways to measure bias:
- Fraction of Perfect Agreement (FPA): The percentage of random seeds where the model's output perfectly matches a specific candidate rule.
- Description Length ($L$): Drawing from Solomonoff’s theory of induction, they use the "prequential code" to measure how many "nats" (units of information) it takes for a model to transmit a dataset. If a dataset following Rule A has a shorter description length than Rule B under a specific model, that model is biased toward Rule A. This is a highly sensitive metric because it captures the "ease" of learning a rule even if the model hasn't fully mastered it yet.
Datasets
The study introduces four new synthetic "quiz-like" tasks designed to test specific types of reasoning:
- Count-or-Memorization: Maps $a^l \to b^l$. Does the model learn to count ($a^m \to b^m$) or just memorize the output length ($a^m \to b^l$)?
- Add-or-Multiply: Maps $a^l \to b^{2l}$. Does it see this as $x + l$ or $2 \cdot x$?
- Hierarchical-or-Linear: Uses nested structures. Does the model pick the middle symbol (hierarchical) or a symbol at a fixed position (linear)?
- Composition-or-Memorization: A simplified version of the SCAN benchmark to see if models prefer combining known primitives or simply memorizing full sequences.
Key Insights and Results
LSTMs are "Natural" Mathematicians Surprisingly, LSTM-based models can infer non-trivial arithmetic rules (counting, addition, multiplication) from a single training example. With a training example of length 40, 100% of LSTMs (without attention) generalized to the "counting" rule. In contrast, Transformers and CNNs almost exclusively preferred "memorization" (FPA > 0.90).
Architecture Dictates Logic There is a fundamental divide in how models "see" structure. Transformers and LSTMs are biased toward hierarchical reasoning (which is essential for human language), whereas CNNs are strongly biased toward linear rules.
As shown in the results for the Hierarchical-or-Linear task, Transformers achieved an FPA of 0.69 for the hierarchical rule (rising to 1.0 with larger embeddings). CNNs, however, showed a perfect 1.0 FPA for the linear rule.
The Simplicity Bias Across all experiments, models consistently followed the "simplest" explanation—defined as the one with the shortest description length. This suggests that description length is a powerful, model-agnostic tool for predicting generalization.
Compositionality On the SCAN benchmark, CNNs showed a strong preference for compositional generalization over memorization as the kernel width increased. Meanwhile, Transformers showed a heavy bias toward memorization in low-data settings.
Limitations and Future Work
- Synthetic Simplicity: The tasks are highly controlled and synthetic. While this eliminates confounding variables, it remains to be seen how these biases interact in complex, real-world natural language datasets.
- The "Why" of CNNs: While the paper identifies that CNNs prefer linear rules, the specific mechanical reason for this preference (beyond the local nature of convolutions) isn't fully dissected.
- Future Direction: The authors suggest this framework could be used to "screen" new architectures for desirable biases (like hierarchy) before training them on massive datasets, or to find ways to inject specific biases into models to improve few-shot learning.
Summary updated