Learning Expressive Meta-Representations with Mixture of Expert Neural Processes

Qi Wang, Herke van HoofNeurIPS 2022 · OpenReview

Introduction

Neural Processes (NPs) are a powerful class of meta-learning models that approximate stochastic processes. Unlike gradient-based methods such as MAML, NPs learn a functional representation of a task, allowing them to adapt to new tasks instantly without requiring further gradient updates at test time.

However, vanilla NPs suffer from a major bottleneck: they typically rely on a single, global Gaussian latent variable to represent an entire task. This "one-size-fits-all" approach struggles when the underlying data generative process is complex or multi-modal—for example, when observations come from a mixture of different stochastic processes. In these scenarios, a single Gaussian cannot capture the intricate functional forms required, leading to underfitting and poor uncertainty quantification. This limitation is particularly detrimental in Meta-Reinforcement Learning (Meta-RL), where accurately modeling uncertainty is crucial for effective exploration.

To resolve these issues, researchers propose Mixture of Expert Neural Processes (MoE-NPs), which significantly boost the expressiveness of standard NPs by replacing the single global latent variable with a mixture of "expert" latent variables.

Methodology: Marrying MoE with Neural Processes

The core innovation of MoE-NPs is integrating Mixture of Experts (MoE) models into the NP framework. Instead of relying on one global latent variable, MoE-NPs introduce two new components:

  1. $K$ Expert Latent Variables ($z_{1:K}$): Multiple continuous variables designed to capture diverse functional priors.
  2. Assignment Latent Variables ($e$): Discrete variables that act as a gating mechanism, determining which expert is responsible for generating a specific data point.
Graphical model of MoE-NPs, illustrating how experts ($z_k$) and assignment variables ($e$) interact to produce observations ($y$).

To make this architecture trainable, the authors utilize variational inference to derive an Evidence Lower Bound (ELBO) objective. The model relies on specialized neural network modules: continuous latent encoders that are permutation-invariant (ensuring the order of context points doesn't alter the task representation), and categorical latent encoders that are permutation-equivariant (ensuring expert assignments remain consistent regardless of the experts' order).

Key Insights and Empirical Results

The authors evaluated MoE-NPs across several benchmarks, including toy regression, few-shot supervised learning (Acrobot system identification, CIFAR-10 image completion), and Meta-RL (2D Navigation, MuJoCo Locomotion).

Capturing Multi-Modality

Standard NPs often "blur" the lines between different underlying processes. By using assignment variables, MoE-NPs can learn "local" properties of the data, assigning different experts to different regions of the input space. This explicit separation provides much more accurate uncertainty estimates and prevents the model from forcing one global variable to explain disjointed phenomena.

Comparison of MoE-NP against standard NPs and other baselines on a mixture of sine and cosine functions. MoE-NP is the only model that clearly identifies the two distinct components.

Performance Gains in Supervised and RL Tasks

  • System Identification: In the Acrobot task (predicting the next state of a double-pendulum system with varying masses), MoE-NPs achieved a Mean Square Error (MSE) of 1.4, significantly outperforming standard NPs (7.2) and strong baselines like MAML (2.5) and CAVIA (2.0).
  • Image Completion: On CIFAR-10, MoE-NPs demonstrated strong asymptotic performance. While gradient-based methods performed well with very few context pixels, MoE-NPs surpassed all baselines as the number of observed pixels increased.
  • Meta-Reinforcement Learning: MoE-NPs demonstrated faster convergence and higher average returns than PEARL (a state-of-the-art context-based Meta-RL method) across several MuJoCo environments. The randomness inherent in the expert latent variables encourages the agent to explore different potential strategies, preventing it from getting stuck in local optima.

Limitations and Future Directions

The primary limitation of MoE-NPs is the difficulty of choosing the optimal number of experts ($K$). Ablation studies revealed a "Goldilocks" problem: while increasing $K$ initially improves performance, having too many experts can degrade it (e.g., in the Acrobot task, performance peaked at $K=5$ and dropped at $K=9$).

Future work aims to address this by incorporating information metrics (like the Bayesian Information Criterion) to automatically determine the number of experts, or by utilizing Hierarchical Dirichlet Processes to allow the model to select the optimal $K$ in a fully Bayesian manner.

Summary updated

Abstract

Neural processes (NPs) formulate exchangeable stochastic processes and are promising models for meta learning that do not require gradient updates during the testing phase. However, most NP variants place a strong emphasis on a global latent variable. This weakens the approximation power and restricts the scope of applications using NP variants, especially when data generative processes are complicated. To resolve these issues, we propose to combine the Mixture of Expert models with Neural Processes to develop more expressive exchangeable stochastic processes, referred to as Mixture of Expert Neural Processes (MoE-NPs). Then we apply MoE-NPs to both few-shot supervised learning and meta reinforcement learning tasks. Empirical results demonstrate MoE-NPs' strong generalization capability to unseen tasks in these benchmarks.
BibTeX
@inproceedings{wang2022learning,
  author = {Qi Wang and Herke van Hoof},
  title = {{Learning Expressive Meta-Representations with Mixture of Expert Neural Processes}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=ju38DG3sbg6},
}
Figure