Human language and thought possess a remarkable property known as systematic compositionality: the ability to understand and produce an infinite number of novel combinations from a finite set of known components. If you know the meaning of the words "dog," "frisbee," "catch," and "throw," you can instantly comprehend the sentence, "The dog threw the frisbee and caught the person," even though the scenario is entirely unfamiliar and factually bizarre.
Modern deep learning models, however, do not naturally share this capability. While they achieve human-level performance on massive benchmarks, they are fundamentally pattern matchers. They prioritize the memorization of fixed statistical patterns within their training distribution rather than acquiring generalizable compositional rules. When these models encounter a test scenario where familiar primitives are arranged in an entirely new structure, their performance drops precipitously. This phenomenon is known as the compositional generalization gap.
The Illusion of Competence
Why does this gap catch us off guard? The primary reason is that standard evaluation metrics hide a model's structural fragility. A neural network can achieve near-perfect accuracy on a large, balanced reasoning dataset, leading researchers to believe it has mastered the underlying logic of a domain.
However, as demonstrated by Dzmitry Bahdanau et al. in their work on the CLOSURE: Assessing Systematic Generalization of CLEVR Models benchmark, high performance is frequently an illusion driven by statistical biases in the training distribution. When the authors recombined familiar visual and textual primitives into novel linguistic structures, the models failed systematically. Instead of treating composite logical structures as predictable interactions of known underlying principles, the networks treated them as independent, completely novel patterns.
This gap persists even in state-of-the-art large language models (LLMs). As Arian Hosseini et al. show in On the Compositional Generalization Gap of In-Context Learning, while scaling up model size and pre-training on code can shrink this gap, LLMs still exhibit a significant performance drop when required to systematically combine basic building blocks into unseen structures during few-shot prompting.
The Anatomy of Failure: Unobserved Local Structures
To build an intuition for why neural networks fail at this task, we must look at how they represent structural relationships. When a model processes a query or generates an output, it maps inputs to outputs through learned representations. If a model has seen every individual concept during training, why does it fail when those concepts are rearranged?
The answer lies in what Ben Bogin et al. term unobserved local structures. In Unobserved Local Structures Make Compositional Generalization Hard, the authors reveal that a model's failure on an out-of-distribution example can be predicted by the appearance of specific, small structural patterns in the target output—such as a novel parent-child or sibling relationship in a logical program—that were never seen during training.
Training Structures Seen:
Structure A: [Modifier 1] -> [Noun 1] (e.g., "red ball")
Structure B: [Modifier 2] -> [Noun 2] (e.g., "plastic cube")
Unseen Test Structure (The Gap):
Structure C: [Modifier 2] -> [Noun 1] (e.g., "plastic ball")
Even if the model perfectly understands Modifier 2 and Noun 1 independently, the local connection between them is entirely unobserved. Because the network treats the composite representation as a holistic pattern rather than a modular assembly, it cannot reliably evaluate or generate the novel combination.
This structural fragility is particularly evident in natural-to-formal language conversion. As observed by Houfeng Wang et al. in Investigating the (De)Composition Capabilities of Large Language Models in Natural-to-Formal Language Conversion, even when models understand individual symbolic operations, they fail to combine them into complex structures that deviate from few-shot demonstrations. The primary bottleneck is not the understanding of the atomic components themselves, but the process of rule discovery and logical assembly.
A Ubiquitous Cross-Domain Challenge
The compositional generalization gap is not an isolated quirk of natural language processing; it is a fundamental architectural limitation that spans nearly every domain of artificial intelligence:
- Open-Domain Question Answering: Models exhibit a sharp performance decline when required to synthesize known semantic components into novel structural combinations, proving that understanding individual entities or predicates does not guarantee the ability to process their novel compositions (Linqing Liu et al., 2022).
- Semantic Parsing and Code Search: Parsers struggle to map natural language to novel SQL query templates because they memorize surface-level alignments rather than learning abstract mapping principles (Tianze Shi et al., 2020). Similarly, code search models fail to interpret familiar programming constructs when they appear in novel combinations (Hojae Han et al., 2022).
- Long-Horizon Robotics: In robotic manipulation, simply increasing the volume of multi-stage training data yields only marginal performance gains. Vision-Language-Action models fail to flexibly recombine atomic physical skills for novel tasks without explicit, grounded hierarchical reasoning (Yangtao Chen et al., 2025).
- Computational Biology: Biological foundation models struggle to interpret how distinct regulatory motifs interact when placed in new combinations, severely limiting their effectiveness in rational sequence design and functional prediction (Ruian Shi et al., 2025).
Pathways to Systematicity
If scaling up parameters and data volume provides only diminishing returns, how can we force neural networks to generalize compositionally? Current research points toward two primary interventions: data augmentation and architectural modularity.
Compositional Data Augmentation
If models fail because they haven't seen specific local structures, we can synthetically expand their training environments to expose them to patterns of interchangeability. Jacob Andreas introduced this concept via Good-Enough Compositional Data Augmentation. By identifying fragments of text that appear in similar environments and automatically swapping them to create synthetic training examples, this protocol explicitly teaches sequence models that known components can be recombined flexibly, drastically reducing out-of-distribution error rates.
Modular Deep Learning
A more permanent fix requires moving away from monolithic "black box" architectures. As argued by Jonas Pfeiffer in Modular Deep Learning, replacing a monolithic network with a collection of specialized, autonomous units allows a system to generalize to entirely new tasks by dynamically recombining learned skills.
By decoupling the learning of atomic operations from the routing logic that connects them, modular architectures prevent parameter interference and mimic the systematic compositionality that characterizes human cognition. Ultimately, bridging the compositional generalization gap requires shifting our training paradigms away from pure statistical pattern matching and toward the explicit optimization of generalizable compositional skills (Brenden Lake, 2023).