Abstract
This paper presents a Deep Mixture of Diverse Experts (DMDE) algorithm designed to scale visual recognition to tens of thousands of classes. By organizing object classes into a two-layer semantic ontology, the framework generates specialized task groups. Each group is handled by a base deep CNN expert trained to recognize specific atomic classes alongside a "not-in-group" category. Through multi-task learning and a novel stacking function for fusion, DMDE effectively leverages inter-class similarities and manages the computational complexities of large-scale datasets like ImageNet10K.
Introduction
Training neural networks for standard benchmarks like ImageNet-1K is a well-understood task. However, scaling recognition to 10,000 or more classes introduces significant hurdles. A monolithic "flat" classifier with a massive softmax layer often suffers from high computational costs, overfitting, and the "confused gradient" problem, where the model struggles to distinguish between highly diverse categories.
The DMDE framework adopts a "divide and conquer" strategy. Instead of one giant network, it utilizes a mixture of specialized experts, each focusing on a semantically related subset of the label space. This approach allows for the use of proven architectures, like AlexNet, as modular building blocks for a much larger system.
Methodology
1. Semantic Ontology and Task Assignment
The label space is organized using a two-layer ontology (category layer and object class layer) derived from WordNet. This ensures that semantically related classes—which often share visual features and learning complexities—are grouped together. Unlike traditional clustering, DMDE employs tree-guided task assignment with significant inter-group overlap. This overlap is crucial for calibrating the experts' confidence scores against one another during the final fusion stage.
2. Specialized Expert Training
Each expert is a CNN trained using a deep multi-task learning (MTL) algorithm. Two key features define the expert training:
- The "Not-in-group" Class: Every expert is trained to recognize its specific classes plus an additional category for objects outside its domain. This acts as a built-in confidence mechanism.
- Multi-Task Softmax: Laplacian regularization is used to ensure that visually similar classes within a group share internal feature representations, enhancing discriminative power.
3. Stacking and Fusion
To produce a final classification, the outputs from all experts are combined via a stacking function. This function uses the "not-in-group" scores to weigh the contributions of each expert; if an expert identifies an image as "not-in-group" with high confidence, its specific class predictions are suppressed in the final output.
Experimental Results
The DMDE framework was evaluated on the ImageNet10K dataset, specifically targeting 7,756 atomic object classes.
Performance Benchmarks
DMDE demonstrated a clear advantage over standard deep learning baselines. On ImageNet10K, it achieved a 38.65% top-1 accuracy, significantly outperforming a standard AlexNet extension (31.70%). The performance gap was even more pronounced in top-K metrics, with DMDE reaching 64.32% top-10 accuracy compared to 52.18% for the baseline.
Component Analysis
Ablation studies highlighted the importance of the multi-task learning and the "not-in-group" safety valve. Removing these components led to a marked decrease in accuracy across the board, confirming that the synergy between semantic grouping and specialized training is vital for large-scale success.
Discussion and Conclusion
The Deep Mixture of Diverse Experts offers a robust framework for large-scale visual recognition by leveraging semantic hierarchies to simplify the learning task. While the current implementation relies on AlexNet, the modular nature of DMDE allows for the integration of more advanced base networks. Future research may focus on optimizing the inference speed of multiple parallel experts and exploring visual-similarity-based ontologies to further refine task grouping.
Summary updated