Extracting Uncertainty Estimates from Mixtures of Experts for Semantic Segmentation

Introduction

In safety-critical domains such as autonomous driving, quantifying a model's uncertainty is as vital as the accuracy of its predictions. While Deep Ensembles provide robust uncertainty estimates, their high computational cost often precludes real-time deployment. This paper explores Mixture of Experts (MoE) architectures as a computationally efficient alternative that provides high-quality, well-calibrated uncertainty estimates "for free."

Methodology

The authors investigate MoE models where a gating network dynamically routes inputs to specialized sub-networks (experts). For semantic segmentation, this allows for experts specialized in different environments, such as highway versus urban scenes.

Expand

Source, p. 4

Uncertainty is extracted using four primary metrics:

1. Predictive Entropy (PE): Captures total uncertainty by measuring the randomness in the final prediction.
2. Mutual Information (MI): Targets epistemic uncertainty by quantifying disagreement between experts.
3. Expert Variance (EV): A novel metric measuring pixel-wise variance across individual expert predictions.
4. Routing Uncertainty: Derived from Gate Entropy, indicating the gating network's indecision.

Experimental Setup

The study utilizes the A2D2 and Cityscapes datasets to evaluate performance across semantically disjoint and overlapping expert configurations. Robustness is further tested using "Corrupted A2D2," which applies 19 types of environmental noise (e.g., blur, snow) to simulate real-world data shifts.

Key Results and Insights

MoEs demonstrate superior performance in Out-of-Distribution (OOD) scenarios compared to traditional ensembles. They are notably better at "knowing what they don't know," achieving PAvPU (Patch Accuracy vs. Patch Uncertainty) scores of up to 100% in specific OOD tests.

Expand

Source, p. 1

Key findings include:

• Zero-overhead Uncertainty: Metrics can be extracted from pre-trained MoEs without additional inference passes or specific training regimes.
• Spatial Awareness: Qualitative results show that MoE uncertainty naturally spikes at object boundaries (aleatoric uncertainty) and in regions where the scene type shifts (epistemic uncertainty).
• Gating Trade-offs: While class-wise gating improves segmentation accuracy (mIoU), simpler gating mechanisms often yield better-calibrated routing uncertainty.

Conclusion

MoEs offer a compelling path toward efficient and reliable uncertainty quantification in semantic segmentation. While increasing the number of experts improves probabilistic fit (NLL), future work is needed to refine gating strategies to prevent calibration plateaus and further leverage expert diversity for more potent epistemic signals.