Introduction
Modern artificial intelligence has trended toward "foundation models"—massive, monolithic architectures with billions of parameters. While powerful, these models are computationally expensive to train and prone to "catastrophic forgetting" in continual learning (CL) scenarios, where new knowledge often overwrites previously learned information. This paper introduces Expert Composition (EC), a modular alternative that leverages specialized, small-scale models to achieve superior performance with significantly lower resource requirements.
The Expert Composition Framework
The core philosophy of EC is compositionality: the ability to understand complex scenes by decomposing them into simpler, familiar parts. Instead of one giant network, the authors propose a two-stage pipeline.
1. Expert Training
The system begins by training a library of specialized "experts." In this study, seven experts were developed using the ResNet-18 architecture. Each expert is trained on a narrow subset of data—specifically, three object classes from the GQA (Genome Question Answering) dataset. A critical component of this training is the inclusion of a generic "other" class. This allows an expert to recognize when an input falls outside its specific domain, effectively acting as a filter during the composition phase.
2. Zero-Shot Composition
To process complex, multi-object images, the EC framework employs a zero-shot approach. When presented with a composite image (such as a 2x2 grid from the CGQA benchmark), the system extracts quadrants and passes them through the expert library. Each expert provides a prediction, and these individual "votes" are concatenated to form the final multi-label output.
Because the experts are never retrained on the composite tasks, they are immune to catastrophic forgetting. This modularity ensures that the system retains its specialized knowledge indefinitely.
Experimental Results
The researchers evaluated EC against state-of-the-art CL baselines using the Compositional GQA (CGQA) benchmark. CGQA specifically tests a model's ability to recognize novel combinations of objects it has seen individually.
Performance and Accuracy
The EC method achieved a remarkable 58% accuracy on the CGQA benchmark. This nearly doubles the performance of traditional CL algorithms like Experience Replay (ER), which reached only 26%. Other common baselines, such as Elastic Weight Consolidation (EWC) and Learning without Forgetting (LwF), struggled significantly, with accuracies ranging between 10% and 20%.
Computational Efficiency
Beyond accuracy, the EC approach offers drastic improvements in sustainability and speed. Training a single expert required approximately 13 minutes on an NVIDIA Tesla T4 GPU. In contrast, standard monolithic CL baselines required nearly 4 hours of training time. Furthermore, because EC is task-agnostic and zero-shot, it achieves peak performance from the very first "experience," eliminating the need for the extensive data exposure required by traditional models.
Conclusion
Expert Composition represents a shift away from the "bigger is better" paradigm in AI. By prioritizing modularity and specialized knowledge, the authors demonstrate that smaller, task-agnostic models can be composed to solve complex problems more effectively than monolithic structures. While currently tested on synthetic datasets, the success of EC suggests a promising path toward more efficient, sustainable, and robust continual learning systems. Future research will focus on scaling this approach to more realistic visual environments and automating the selection of experts from large-scale model repositories.
Summary updated