Flexible and efficient simulation-based inference for models of decision-making

Jan Boelts, Jan-Matthis Lueckmann, Richard Gao, Jakob H. MackeOpenReview 2022 · OpenReview

TL;DR

This paper introduces Mixed Neural Likelihood Estimation (MNLE), a new method for efficiently inferring parameters of complex computational models used in cognitive neuroscience. MNLE uses neural networks to learn an "emulator" of the model, allowing for fast and accurate Bayesian inference even when the model's likelihood function is intractable, and it achieves this with significantly fewer simulations than previous methods.

What Problem Does This Paper Address?

Cognitive neuroscience relies heavily on computational models to understand decision-making. A key step is to "fit" these models to experimental data by estimating their parameters. While simple optimization can find the single best-fitting parameters, Bayesian inference offers a richer understanding by quantifying uncertainty and exploring all plausible parameter values consistent with the data and prior knowledge. However, a major hurdle is that many complex models, especially those used in neuroscience, have likelihood functions that are analytically intractable or computationally prohibitive to evaluate. This makes standard Bayesian inference methods like Markov Chain Monte Carlo (MCMC) infeasible. Simulation-Based Inference (SBI) offers a solution by only requiring the ability to simulate from the model, but existing SBI methods can be computationally expensive and inflexible, often requiring retraining for slight model or data variations.

Methods

The paper introduces Mixed Neural Likelihood Estimation (MNLE). MNLE is a type of Simulation-Based Inference (SBI) that uses neural networks to learn an emulator of the computational model. Instead of directly estimating the likelihood, MNLE trains a neural network to predict the probability of observing certain data given a set of model parameters.

Here's how it works:

  1. Training Data Generation: A set of model parameters is sampled from a proposal distribution (often the prior used in inference). For each parameter set, the computational model is run to generate simulated data.
  2. Neural Density Estimation: MNLE uses a conditional neural density estimator to learn the relationship between parameters and simulated data. Crucially, it's designed to handle mixed data types (e.g., binary choices and continuous reaction times), common in decision-making experiments. It achieves this by factorizing the likelihood into discrete and continuous components, using separate neural networks for each.
  3. Bayesian Inference: Once trained, the neural network emulator can provide approximate likelihoods for new, observed data. These approximate likelihoods can then be used with standard MCMC methods to perform Bayesian parameter inference.

The key novelty lies in MNLE's ability to handle mixed data types efficiently and its simulation efficiency compared to prior methods like Likelihood Approximation Networks (LANs).

Datasets

The paper uses simulated data from two variants of the Drift-Diffusion Model (DDM) for training and evaluation. The DDM is a standard model in cognitive neuroscience for describing decision-making tasks. The authors compare their method to LANs using a publicly available pre-trained LAN model. No new benchmark datasets were introduced.

Key Insights and Ideas

  • Bridging the Gap for Mixed Data: Decision-making experiments often produce data that is a mix of discrete (e.g., choice: left/right) and continuous (e.g., reaction time) variables. MNLE's architecture is specifically designed to handle this complexity, which is a significant advantage over methods that primarily focus on continuous data.
  • Simulation Efficiency is King: The most striking insight is MNLE's dramatic reduction in the number of model simulations required for training. This makes complex Bayesian inference accessible to researchers who may not have access to massive computational resources.
  • Emulator Power: The trained neural network acts as a "probabilistic emulator" of the simulator. This means it can not only provide likelihoods but also generate synthetic data, which is useful for model checking and understanding the emulator's behavior.
  • Flexibility Beyond Canonical Models: Unlike some SBI methods that are tied to specific, well-studied models (like the DDM), MNLE's framework is general and can be applied to any simulation-based model with intractable likelihoods, encouraging the development and testing of novel computational models.

Key Results

  • Accuracy: MNLE achieved comparable, and in some metrics slightly better, accuracy in approximating the DDM's likelihoods compared to LANs.
  • Simulation Budget: MNLE required six orders of magnitude fewer model simulations for training than LANs (10⁵ vs. >10¹¹). This is the most significant practical advantage highlighted.
  • Evaluation Speed: The time taken to evaluate the learned likelihoods for MCMC inference was comparable between MNLE and LANs.
  • Posterior Recovery and Calibration: MNLE produced posteriors that were as accurate as, and in some aspects better calibrated than, those obtained using LANs. This was demonstrated through parameter recovery tests and simulation-based calibration (SBC).
  • Intractable Models: MNLE was successfully applied to a variant of the DDM with collapsing bounds, for which analytical likelihoods are unavailable, demonstrating its utility for truly intractable models.

Limitations and Future Work

  • Hyperparameter Tuning: While MNLE builds on existing frameworks, it still requires careful selection of hyperparameters for the neural network architecture. The authors mention using cross-validation for this, but acknowledge that this often relies on access to analytical likelihoods, which is precisely what MNLE aims to circumvent.
  • Computational Cost of Normalizing Flows: The paper notes that the use of complex normalizing flows for the continuous data component can lead to longer evaluation times, though they found an architecture that matched LANs' performance.
  • Future Directions:
    • Improving simulation efficiency further by using weight sharing between discrete and continuous density estimators.
  • Applying MNLE to high-dimensional inference problems where MCMC might struggle, potentially in conjunction with variational inference.
  • Utilizing the emulator property in active learning settings for expensive simulators.
  • Adapting MNLE for scenarios with stimulus- and inter-trial dependencies, or where the i.i.d. assumption across trials does not hold.

Summary updated

Abstract

Identifying parameters of computational models that capture experimental data is a central task in cognitive neuroscience. Bayesian statistical inference aims to not only identify a single configuration of best-fitting parameters, but to recover all model parameters that are consistent with the data and prior knowledge. Statistical inference methods usually require the ability to evaluate the likelihood of the model—however, for many models of interest in cognitive neuroscience, the associated likelihoods cannot be computed efficiently. Simulation-based inference (SBI) offers a solution to this problem by only requiring access to simulations produced by the model. Here, we provide an efficient SBI method for models of decision-making. Our approach, Mixed Neural Likelihood Estimation (MNLE), trains neural density estimators on model simulations to emulate the simulator. The likelihoods of the emulator can then be used to perform Bayesian parameter inference on experimental data using standard approximate inference methods like Markov Chain Monte Carlo sampling. While most neural likelihood estimation methods target continuous data, MNLE works with mixed data types, as typically obtained in decision-making experiments (e.g., binary decisions and associated continuous reaction times). We demonstrate MNLE on two variants of the drift-diffusion model (DDM) and compare its performance to a recently proposed method for SBI on DDMs, called likelihood approximation networks (LAN, Fengler et al. 2021). We show that MNLE is substantially more efficient than LANs, requiring six orders of magnitudes fewer model simulations to achieve comparable likelihood accuracy and evaluation time while providing the same level of flexibility. We include an implementation of our algorithm in the user-friendly open-source package sbi.
BibTeX
@misc{boelts2022flexible,
  author = {Jan Boelts and Jan-Matthis Lueckmann and Richard Gao and Jakob H. Macke},
  title = {{Flexible and efficient simulation-based inference for models of decision-making}},
  year = {2022},
  url = {https://openreview.net/forum?id=9B8DEk2W3bn},
  howpublished = {OpenReview},
}
Figure