Abstract
We describe nonparametric deconvolution models (NDMs), a family of Bayesian nonparametric models for collections of data in which each observation is the average over the features from heterogeneous particles. For example, these types of data are found in elections, where we observe precinct-level vote tallies (observations) of individual citizens' votes (particles) across each of the candidates or ballot measures (features), where each voter is part of a specific voter cohort or demographic (factor). Like the hierarchical Dirichlet process, NDMs rely on two tiers of Dirichlet processes to explain the data with an unknown number of latent factors; each observation is modeled as a weighted average of these latent factors. Unlike existing models, NDMs recover how factor distributions vary locally for each observation. This uniquely allows NDMs both to deconvolve each observation into its constituent factors, and also to describe how the factor distributions specific to each observation vary across observations and deviate from the corresponding global factors. We present variational inference techniques for this family of models and study its performance on simulated data and voting data from California. We show that including local factors improves estimates of global factors and provides a novel scaffold for exploring data.
The Challenge of Convolved Data
In many scientific and social domains, researchers only observe "convolved" data—averages or aggregates of many individual "particles." Standard analytical tools like Principal Component Analysis (PCA) or Latent Dirichlet Allocation (LDA) attempt to decompose these aggregates into global "factors." However, these methods typically assume that a factor (e.g., a specific voter demographic or cell type) looks identical across all observations.
In reality, local context matters. A "middle-class voter" may prioritize different issues depending on their precinct. The challenge lies in deconvolving these aggregates while explicitly modeling local deviations from global patterns, all without knowing the number of latent factors in advance. To understand how these concepts translate across domains, consider the following mapping:
Methods: Introducing Local Factor Features
NDMs build upon the foundation of the Hierarchical Dirichlet Process (HDP) but introduce a critical innovation: local factor features. While standard mixture models assign an observation to a single group and admixture models allow an observation to be a mixture of groups, NDMs allow the groups themselves to vary locally.
As shown in the comparison above, the NDM framework (bottom) introduces a variable that represents the average feature values for a specific factor within a specific observation. This allows the model to capture how a group's behavior shifts locally. To handle various data types (counts, proportions, real-valued data), NDMs utilize link functions similar to Generalized Linear Models. For inference, the authors employ Variational Inference (VI) combined with "split and merge" operations, allowing the model to automatically discover the optimal number of latent factors.
Key Results and Insights
Superior Recovery and Global Estimation
Counter-intuitively, allowing the model to "soak up" local variation into local parameters actually makes the estimates for the global factors cleaner and more accurate. In simulations across various data domains, NDMs consistently outperformed standard methods like PCA and Non-negative Matrix Factorization (NMF) in recovering both global means and factor proportions.
Discovery in California Voting Data
When applied to 2016 California precinct-level voting data, the NDM discovered 10 latent voting cohorts. Beyond identifying general trends, the model revealed geographic nuances. For example, it identified a specific cohort that was generally opposed to marijuana legalization but found that members of that same cohort living near Death Valley actually favored it—a local deviation likely tied to regional concerns regarding illegal cultivation sites.
Conclusion and Future Work
Nonparametric Deconvolution Models provide a powerful framework for unmixing aggregate data while preserving the privacy of individual particles. By identifying where local behavior deviates from the global norm, NDMs provide a "scaffold" for data exploration, pointing researchers toward regional anomalies. Future work may involve integrating external metadata (such as census data) directly into the model to further guide the deconvolution process and improving the computational scalability of the split-merge operations for even larger datasets.
Summary updated