Variational Autoencoders for Highly Multivariate Spatial Point Processes Intensities

Baichuan Yuan, Xiaowei Wang, Jianxin Ma, Chang Zhou, Andrea L. Bertozzi, Hongxia YangICLR 2020 · OpenReview

Abstract

Multivariate spatial point process models are essential for describing heterotopic data over space. However, highly multivariate intensities pose significant computational challenges due to the curse of dimensionality. To bridge this gap, this paper introduces a declustering-based hidden variable model that enables efficient inference via a Variational Autoencoder (VAE). The authors demonstrate that this model is a generalization of the VAE-based model for collaborative filtering, opening up novel applications of spatial point process models to recommender systems.

The Challenge of Highly Multivariate Data

Spatial Point Processes (SPP) serve as the mathematical foundation for modeling events occurring in a continuum—such as user check-ins in a city or the locations of trees in a forest. However, when dealing with "highly multivariate" data, like millions of users interacting within a social network, traditional SPP methods like Gaussian Processes (GP) hit a computational wall. They suffer from the "curse of dimensionality" and become prohibitively expensive to scale.

Furthermore, real-world data is often "heterotopic," meaning it is missing in some areas for some users. Existing models struggle to jointly infer the hidden correlations between thousands of subprocesses to predict events in unobserved areas. A scalable, non-parametric method is needed—one that doesn't assume a rigid functional form, scales to millions of dimensions, and handles missing data gracefully.

The VAE-SPP Framework

To address these limitations, the authors propose VAE-SPP, a model that treats the intensity of a point process as a realization of a latent variable model.

VAE-SPP Architecture
  1. Nonparametric Intensity: Instead of relying on a fixed parametric form, the model extends the classic kernel estimator. The intensity for a subprocess (e.g., a user) $u$ is modeled as a weighted sum of kernels centered at all observed event locations: $\lambda_u(x) = \sum p_i^u K_\sigma(x - x_i)$.
  2. Amortized Inference (VAE): To avoid the massive cost of optimizing parameters for every single user, the framework utilizes a Variational Autoencoder. An encoder network maps a user's observed events to a latent distribution $z_u$. A decoder network then maps $z_u$ to the weights $p_i^u$ for the intensity function.
  3. The "Alternative" Model: The authors prove a theoretical equivalence between a standard Poisson process and a model that samples a fixed number of points from a normalized density. This bypasses difficult normalization terms during training, making gradient descent significantly more efficient.
  4. Latent Space Embedding: For datasets lacking physical coordinates (like movie ratings), Graph Neural Networks (GNNs) are used to embed items into a latent spatial continuum. This allows the SPP framework to be applied seamlessly to general recommendation tasks.

Key Insights: Bridging Spatial Statistics and Collaborative Filtering

One of the paper's most striking theoretical contributions is the proof that state-of-the-art VAE-based collaborative filtering (VAE-CF) is actually a special case of this SPP model. Specifically, VAE-CF emerges if you use a delta function as your kernel.

By moving from a discrete delta function to a smoother kernel (like an RBF kernel), the model gains the ability to generalize across space. It doesn't just learn that a user likes "Item A"; it learns the user likes the "area" around Item A in the latent or physical space. This architecture is inspired by "stochastic declustering," a seismology technique used to separate background noise from triggered earthquakes, repurposed here to handle latent user preferences.

Experimental Results

The model was evaluated on synthetic datasets (Multiexponential and Multisine), physical spatial data (Gowalla check-ins in NYC and California), and latent spatial data (MovieLens benchmarks).

  • Superior Accuracy: VAE-SPP consistently outperformed VAE-CF, standard Kernel Density Estimation (KDE), and Trans-Gaussian Cox Processes (TGCP) across key metrics like NDCG and Recall. For instance, on the Gowalla NYC dataset, VAE-SPP achieved an NDCG@100 of 44.8 compared to 43.6 for VAE-CF.
  • Efficiency: Thanks to amortized inference, once the model is trained, predicting the intensity for a new user is near-instantaneous, a massive advantage over GP-based methods.
  • Visual Smoothness: Qualitative results demonstrate that VAE-SPP produces much more realistic and continuous intensity maps than discrete models, which tend to produce "spiky" or overfitted predictions.
Qualitative comparison of intensity maps

Limitations and Future Work

While powerful, VAE-SPP has a few limitations. Like most VAE-based approaches, it is data-hungry and requires a large amount of training data to learn latent correlations effectively. Additionally, the "alternative model" used for training efficiency cannot currently infer the total number of missing events—only their relative distribution (where they are most likely to occur).

Future directions suggested by the authors include extending the framework to spatiotemporal processes by adding the dimension of time, and incorporating external features (covariates) into the intensity function to further enhance predictive power.

Summary updated

Abstract

Multivariate spatial point process models can describe heterotopic data over space. However, highly multivariate intensities are computationally challenging due to the curse of dimensionality. To bridge this gap, we introduce a declustering based hidden variable model that leads to an efficient inference procedure via a variational autoencoder (VAE). We also prove that this model is a generalization of the VAE-based model for collaborative filtering. This leads to an interesting application of spatial point process models to recommender systems. Experimental results show the method's utility on both synthetic data and real-world data sets.
BibTeX
@inproceedings{yuan2020variational,
  author = {Baichuan Yuan and Xiaowei Wang and Jianxin Ma and Chang Zhou and Andrea L. Bertozzi and Hongxia Yang},
  title = {{Variational Autoencoders for Highly Multivariate Spatial Point Processes Intensities}},
  booktitle = {International Conference on Learning Representations},
  year = {2020},
  url = {https://openreview.net/forum?id=C_6C7cwGoT_},
}
Figure