Accurate inference in parametric models reshapes neuroscientific interpretation and improves data-driven discovery

Pratik Sachdeva, Jesse A. Livezey, Maximilian E. Dougherty, Bon-Mi Gu, Joshua D. Berke, Kristofer E. BouchardbioRxiv 2020 · DOI

TL;DR

This paper introduces the Union of Intersections (UoI) framework, a novel statistical inference method that significantly improves the accuracy of parameter estimation in neuroscience models. By applying UoI to functional coupling, encoding, and decoding models across diverse neural datasets, the authors demonstrate that it leads to sparser, more stable, and more interpretable models without sacrificing predictive performance, thereby reshaping our understanding of neural systems.

What Problem Does This Paper Address?

Systems neuroscience aims to understand how neural populations process information, often relying on statistical models to interpret complex, high-dimensional neural recordings. Parametric models are particularly useful because their parameters can offer insights into underlying biological processes. However, traditional methods for fitting these models often struggle with accurately selecting the correct parameters (i.e., identifying which factors are truly influential) and estimating their values precisely. This can lead to inaccurate interpretations of neural function, potentially misrepresenting the importance of certain neural connections or external factors. The authors question whether past neuroscience findings might be flawed due to these inference limitations and propose UoI as a solution.

Methods

The core of this work is the Union of Intersections (UoI) framework. UoI is a flexible approach that combines regularization (to encourage sparsity and feature selection) with ensemble methods (to improve stability and reduce variance). It separates model selection and estimation into two stages:

  1. Model Selection: This stage uses a "compressive" approach. It involves fitting models with varying degrees of regularization (like Lasso) across multiple resamples of the data. The "intersection" step then identifies features that are consistently selected across these resamples, ensuring robustness.
  2. Model Estimation: This stage uses an "expansive" approach. For the selected features from the previous stage, it estimates the model parameters without regularization, using an ensemble of the best-performing models across resamples.

This two-stage process, particularly in its UoILasso variant, aims to achieve accurate parameter selection and estimation, leading to models that are both sparse and stable, while maintaining high predictive performance.

Datasets

The authors applied their UoI framework to a diverse range of neural datasets, including:

  • Auditory Cortex (AC): Rat micro-electrocorticography (µECoG) recordings.
  • Primary Visual Cortex (V1): Macaque single-unit recordings.
  • Primary Motor Cortex (M1): Macaque single-unit recordings.
  • Retina: Mouse single-unit recordings.
  • Basal Ganglia: Rat tetrode recordings.

These datasets cover different brain regions, animal models, and recording modalities, allowing for a comprehensive evaluation of the UoI framework's generalizability.

Key Insights and Ideas

  • Accurate Inference is Crucial for Interpretation: The study strongly suggests that the choice of inference procedure significantly impacts not just model fit, but also the biological conclusions drawn from the data. Poor inference can lead to misleading interpretations of neural circuits.
  • Sparsity Without Predictive Loss: UoI consistently produces sparser models (fewer parameters) across all tested model types and datasets, yet this sparsity does not come at the cost of predictive accuracy. This is a key finding, as it suggests that many previously identified neural relationships might have been spurious.
  • Reshaping Network Structure: The increased sparsity from UoI leads to qualitatively different network structures. Functional coupling networks become more modular and less "small-world," suggesting that the underlying organization of neural communication might be better captured by these sparser representations.
  • Identifying "Non-Tuned" Neurons: In encoding models, UoI can more accurately identify neurons that do not exhibit meaningful tuning to stimuli, preventing false positives that might arise from less precise inference methods. This is crucial for understanding how neurons selectively represent information.
  • Parsimony Matters: The Bayesian Information Criterion (BIC) consistently favored UoI models, indicating they are more parsimonious, meaning they achieve good predictive performance with fewer parameters.

Key Results

  • Reduced Model Complexity: UoI models used 2-5 times fewer parameters than baseline models across functional coupling, encoding, and decoding tasks. For example, in basal ganglia decoding, UoI models achieved perfect classification accuracy using only 2 neurons, compared to 8 for baseline models.
  • Maintained Predictive Performance: Despite the significant reduction in model complexity, predictive performance (measured by R², deviance, or classification accuracy) was largely maintained or only negligibly affected. Statistical tests showed differences were often statistically significant but with very small effect sizes, indicating they were not practically relevant.
  • Enhanced Network Modularity: Functional coupling networks inferred by UoI showed significantly larger modularity compared to baseline models, indicating a stronger community structure.
  • Decreased Small-Worldness: UoI networks were considerably less small-world than baseline networks, suggesting a different organizational principle for neural communication.
  • Improved Parameter Estimates: UoI models exhibited reduced shrinkage (less bias) and fewer false positives in parameter estimates compared to baseline methods.

Limitations and Future Work

  • Generalizability to Other Model Classes: The authors acknowledge that their study focused on generalized linear models. They suggest that the principles of encouraging stability and sparsity, as employed by UoI, could potentially be extended to other model classes, such as artificial neural networks (ANNs) or dimensionality reduction methods, which are also widely used in neuroscience.
  • Exploration of Other Biological Contexts: While the study focused on neuroscience, the authors note that model-based data-driven discovery is prevalent across biology. They hypothesize that similar benefits of accurate parameter inference could be seen in other biological fields.
  • Model Selection Criteria: The paper discusses that there isn't a single universally preferred model selection criterion. While BIC favored UoI, other criteria might yield slightly different results, though the overall trend of improved parsimony and interpretability is expected to hold.
  • Future Work: The authors propose investigating the impact of UoI on models that capture neural dynamics (e.g., vector auto-regressive models) and exploring its application to other biological contexts. They also suggest assessing how UoI's principles might apply to complex models like ANNs.

Summary updated

Abstract

Abstract A central goal of systems neuroscience is to understand the relationships amongst constituent units in neural populations and their modulation by external factors using high-dimensional and stochastic neural recordings. Statistical models, particularly parametric models, play an instrumental role in accomplishing this goal, because their fitted parameters can provide insight into the underlying biological processes that generated the data. However, extracting conclusions from a parametric model requires that it is fit using an inference procedure capable of selecting the correct parameters and properly estimating their values. Traditional approaches to parameter inference have been shown to suffer from failures in both selection and estimation. Recent development of algorithms that ameliorate these deficiencies raises the question of whether past work relying on such inference procedures have produced inaccurate systems neuroscience models, thereby impairing their interpretation. Here, we used the Union of Intersections, a statistical inference framework capable of state-of-the-art selection and estimation performance, to fit functional coupling, encoding, and decoding models across a battery of neural datasets. We found that, compared to baseline procedures, UoI inferred models with increased sparsity, improved stability, and qualitatively different parameter distributions, while maintaining predictive performance across recording modality, brain region, and task. Specifically, we obtained highly sparse functional coupling networks with substantially different community structure, more parsimonious encoding models, and decoding models that rely on fewer single-units. Together, these results demonstrate that accurate parameter inference reshapes interpretation in diverse neuroscience contexts. The ubiquity of model-based data-driven discovery in biology suggests that analogous results would be seen in other fields.
BibTeX
@misc{sachdeva2020accurate,
  author = {Pratik Sachdeva and Jesse A. Livezey and Maximilian E. Dougherty and Bon-Mi Gu and Joshua D. Berke and Kristofer E. Bouchard},
  title = {{Accurate inference in parametric models reshapes neuroscientific interpretation and improves data-driven discovery}},
  year = {2020},
  doi = {10.1101/2020.04.10.036244},
  howpublished = {bioRxiv},
}
Figure