What they do when in doubt: a study of inductive biases in seq2seq learners

Eugene Kharitonov, Rahma Chaabouni

TL;DR

When faced with ambiguous data, different neural architectures (LSTMs, CNNs, and Transformers) exhibit strikingly different "hidden" preferences, or inductive biases. By measuring "description length"—how easily a model can represent a rule—researchers found that LSTMs can learn to count or add from a single example, while Transformers and LSTMs inherently prefer hierarchical rules over linear ones, unlike CNNs.

The Problem: Poverty of the Stimulus

In machine learning, we often encounter the "poverty of the stimulus": training data is too sparse to point to a single correct way to generalize. For example, if a model learns that $f(3) = 6$, should it assume $f(4)$ is $7$ (add 3), $8$ (multiply by 2), or $6$ (a constant function)?

While humans have strong innate biases—like assuming language has a hierarchical structure—we don't fully understand the "gut instincts" of our sequence-to-sequence (seq2seq) models. This paper maps out these inductive biases to understand why certain architectures succeed or fail at specific tasks, particularly in low-data regimes.

Methods for Measuring Bias

The authors propose a formal framework to "interrogate" models by training them on highly ambiguous, minimal datasets (often just one example) and observing which rule they follow on new inputs.

They introduce two primary ways to measure bias:

  1. Fraction of Perfect Agreement (FPA): The percentage of random seeds where the model's output perfectly matches a specific candidate rule.
  2. Description Length ($L$): Drawing from Solomonoff’s theory of induction, they use the "prequential code" to measure how many "nats" (units of information) it takes for a model to transmit a dataset. If a dataset following Rule A has a shorter description length than Rule B under a specific model, that model is biased toward Rule A. This is a highly sensitive metric because it captures the "ease" of learning a rule even if the model hasn't fully mastered it yet.

Datasets

The study introduces four new synthetic "quiz-like" tasks designed to test specific types of reasoning:

Four synthetic tasks illustrating ambiguous training data.
  • Count-or-Memorization: Maps $a^l \to b^l$. Does the model learn to count ($a^m \to b^m$) or just memorize the output length ($a^m \to b^l$)?
  • Add-or-Multiply: Maps $a^l \to b^{2l}$. Does it see this as $x + l$ or $2 \cdot x$?
  • Hierarchical-or-Linear: Uses nested structures. Does the model pick the middle symbol (hierarchical) or a symbol at a fixed position (linear)?
  • Composition-or-Memorization: A simplified version of the SCAN benchmark to see if models prefer combining known primitives or simply memorizing full sequences.

Key Insights and Results

LSTMs are "Natural" Mathematicians Surprisingly, LSTM-based models can infer non-trivial arithmetic rules (counting, addition, multiplication) from a single training example. With a training example of length 40, 100% of LSTMs (without attention) generalized to the "counting" rule. In contrast, Transformers and CNNs almost exclusively preferred "memorization" (FPA > 0.90).

Architecture Dictates Logic There is a fundamental divide in how models "see" structure. Transformers and LSTMs are biased toward hierarchical reasoning (which is essential for human language), whereas CNNs are strongly biased toward linear rules.

FPA and Description Length (L) for Hierarchical vs. Linear rules across architectures.

As shown in the results for the Hierarchical-or-Linear task, Transformers achieved an FPA of 0.69 for the hierarchical rule (rising to 1.0 with larger embeddings). CNNs, however, showed a perfect 1.0 FPA for the linear rule.

The Simplicity Bias Across all experiments, models consistently followed the "simplest" explanation—defined as the one with the shortest description length. This suggests that description length is a powerful, model-agnostic tool for predicting generalization.

Compositionality On the SCAN benchmark, CNNs showed a strong preference for compositional generalization over memorization as the kernel width increased. Meanwhile, Transformers showed a heavy bias toward memorization in low-data settings.

Limitations and Future Work

  • Synthetic Simplicity: The tasks are highly controlled and synthetic. While this eliminates confounding variables, it remains to be seen how these biases interact in complex, real-world natural language datasets.
  • The "Why" of CNNs: While the paper identifies that CNNs prefer linear rules, the specific mechanical reason for this preference (beyond the local nature of convolutions) isn't fully dissected.
  • Future Direction: The authors suggest this framework could be used to "screen" new architectures for desirable biases (like hierarchy) before training them on massive datasets, or to find ways to inject specific biases into models to improve few-shot learning.

Summary updated

Abstract

Sequence-to-sequence (seq2seq) learners are widely used, but we still have only limited knowledge about what inductive biases shape the way they generalize. We address that by investigating how popular seq2seq learners generalize in tasks that have high ambiguity in the training data. We use four new tasks to study learners' preferences for memorization, arithmetic, hierarchical, and compositional reasoning. Further, we connect to Solomonoff's theory of induction and propose to use description length as a principled and sensitive measure of inductive biases. In our experimental study, we find that LSTM-based learners can learn to perform counting, addition, and multiplication by a constant from a single training example. Furthermore, Transformer and LSTM-based learners show a bias toward the hierarchical induction over the linear one, while CNN-based learners prefer the opposite. The latter also show a bias toward a compositional generalization over memorization. Finally, across all our experiments, description length proved to be a sensitive measure of inductive biases.
BibTeX
@inproceedings{kharitonov2021what,
  author = {Eugene Kharitonov and Rahma Chaabouni},
  title = {{What they do when in doubt: a study of inductive biases in seq2seq learners}},
  booktitle = {International Conference on Learning Representations},
  year = {2021},
  url = {https://openreview.net/forum?id=YmA86Zo-P_t},
}
Figure