Zero-Shot Generalization via Modular Composition

34 concepts28 papers1 proposal

Deep learning models excel when interpolating within their training distribution, but they notoriously struggle when faced with novel combinations of known concepts. For example, a model trained to solve vehicle routing problems with time windows, and separately trained to solve problems with capacity constraints, will often fail when asked to solve a problem containing both constraints simultaneously. This is the challenge of combinatorial generalization.

Instead of training massive, monolithic architectures that attempt to map the entire joint distribution of all possible constraints—a strategy that hits a wall of diminishing returns in cost and data efficiency as argued by Yilun Du et al. in Position: Compositional Generative Modeling: A Single Model is Not All You Need—a growing body of research focuses on modular composition. The core intuition is simple: if we can factorize a problem space into independent, reusable neural modules that each master a single atomic attribute or operator, we can synthesize solutions to entirely unseen tasks at inference time by simply recombining these modules.

Factorizing the Problem Space

To achieve zero-shot generalization, a system must first disentangle the constituent parts of a task. Rather than learning a single end-to-end mapping, the architecture isolates different dimensions of variation into independent components.

In combinatorial optimization, such as the Vehicle Routing Problem (VRP), complex variants are broken down into independent "attribute blocks." For instance, Han-Seul Jeong et al. in ARC: Leveraging Compositional Representations for Cross-Problem Learning on VRPs disentangle the intrinsic meaning of a constraint from its contextual interaction. Similarly, Fei Liu 0044 et al. in Multi-Task Learning for Routing Problem with Cross-Problem Zero-Shot Generalization, Jianan Zhou et al. in MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts, and Zikang Yu et al. in Combination-of-Experts with Knowledge Sharing for Cross-Task Vehicle Routing Problems demonstrate that treating constraints like capacity or time windows as modular features allows a single neural solver to address novel constraint permutations zero-shot.

This factorization applies equally to language and domain adaptation. As surveyed by Jonas Pfeiffer et al. in Modular Deep Learning, decoupling task-specific logic from domain-specific features allows a model to execute a task in an entirely new domain or language. For example, Jerin Philip et al. in Monolingual Adapters for Zero-Shot Neural Machine Translation substitute language-pair adapters with independent monolingual modules that map inputs into a shared latent space, bridging unseen language pairs.

In the realm of large language models, Junhong Shen et al. in Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domains train domain and function modules independently, enabling the model to apply specialized functions to completely novel domains zero-shot. Similarly, Zhangyin Feng in SkillNet-X: A Multilingual Multitask Model with Sparsely Activated Skills shows that sparsely activating reusable skill modules allows for the rapid synthesis of new task-language capabilities.

Mechanisms of Inference-Time Composition

How do these independent modules combine without further training or backpropagation? Researchers employ several distinct mechanisms to compose modules on the fly:

Algebraic and Energy-Based Summation

One elegant approach is to represent constraints as independent energy functions or potentials. Armand Comas et al. demonstrate in Inferring Relational Potentials in Interacting Systems that multi-agent interactions can be modeled as separate energy terms, which are combined via simple summation at inference time to handle hybrid environments. For visual generation, the authors of Product of Experts for Visual Generation use an annealed sampling strategy to combine specialized experts (e.g., physics simulators and pixel generators) at inference time, achieving high controllability without retraining.

Dynamic Routing and Feature-Based Selection

Instead of static combination, models can dynamically route inputs through a library of modules based on context. Nasim Rahaman et al. in Dynamic Inference with Neural Interpreters use self-attention to route data through a learned sequence of independent functions, improving systematic generalization on abstract tasks.

In continual learning settings, Oleksiy Ostapenko et al. in Continual Learning via Local Module Composition equip modules with local structural components that self-determine relevance based on input features rather than explicit task labels, enabling zero-shot out-of-distribution generalization. For few-shot learning, Tianxiang Sun et al. in Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning pre-train a library of modular prompts and use a router to combine them for unseen tasks.

Symbolic and Pipeline Orchestration

Complex tasks can be broken down into executable programs or pipelines of foundation models. Joy Hsu et al. in NS3D: Neuro-Symbolic Grounding of 3D Objects and Relations translate natural language into symbolic structures that recombine pre-trained functional neural modules, allowing zero-shot transfer from object grounding to question answering. Similarly, Junsheng Zhou et al. in Zero-Shot Scene Reconstruction from Single Images with Deep Prior Assembly orchestrate a pipeline of specialized vision models (segmentation, inpainting, 3D generation) to generalize to out-of-distribution 3D scenes.

Parameter-Space Assembly

Modules can also be physically or architecturally assembled. Jiacong Hu et al. in Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks introduce a framework to strip and snap together task-specific components into new architectures without backpropagation. In object tracking, Gianluca Mancusi et al. in Is Multiple Object Tracking a Matter of Specialization? train lightweight experts for specific scene attributes and merge them directly in parameter space to adapt to novel domains.

Why It Works: The Intuition of Functional Isolation

Why does modular composition succeed where monolithic fine-tuning fails? The answer lies in functional isolation and the avoidance of negative interference.

In a standard monolithic model, updating weights to satisfy a new constraint alters a shared parameter space, often degrading performance on previously learned rules—a phenomenon known as catastrophic forgetting or gradient interference. By contrast, modular architectures isolate the learning of individual atomic operators. As Michael Chang et al. demonstrate in Automatically Composing Representation Transformations as a Means for Generalization, training a system on a library of atomic operators allows it to discover new sequences of existing modules to solve novel task combinations without requiring joint training data.

By reconfiguring a library of pre-trained modules to match the structure of an unseen problem, agents can achieve immediate success. Jorge A. Mendez et al. show in Modular Lifelong Reinforcement Learning via Neural Composition that this functional compositionality eliminates catastrophic forgetting and enables zero-shot generalization to novel task combinations. This is echoed in robotics by Runhan Huang et al. in MoE-Loco: Mixture of Experts for Multitask Locomotion, where manually combining independent expert modules (such as balancing and agility) yields complex, novel behaviors zero-shot.

Furthermore, structured configurations allow the model to explicitly map out how to combine these isolated skills. Jifan Chen et al. show in Improving Cross-task Generalization of Unified Table-to-text Models with Compositional Task Configurations that decomposing complex tasks into explicit configurations (input format, output type, task category) allows models to treat operations like evidence retrieval and summarization as independent, reusable capabilities, drastically improving zero-shot performance on unseen tasks.

Ultimately, modular composition works because it shifts the generalization burden from statistical interpolation (hoping the model saw a similar joint distribution during training) to structural reconfiguration (assembling known, robust components to satisfy a new set of constraints).

Key Foundational Work

Automatically Composing Representation Transformations as a Means for Generalization (2019) — contributed 1 concept elements to this direction.
Monolingual Adapters for Zero-Shot Neural Machine Translation (2020) — contributed 1 concept elements to this direction.
Compositional Fairness Constraints for Graph Embeddings (2019) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_combinatorial_generalization_a115f414,
  title = {combinatorial generalization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/combinatorial-generalization/rdr_a115f414f46e46dea6c3b5e5eacbe26d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_combinatorial_generalization_a115f414_v1,
  title = {combinatorial generalization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/combinatorial-generalization/rdr_a115f414f46e46dea6c3b5e5eacbe26d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake