Modeling Attraction and Repulsion with Nonsymmetric Kernels

This direction is split into other directions: negative correlation constraint. Its last version stays readable and citable here.

Version 1 of 118 papers35 conceptsfrozen 2026-10-04

Traditional probabilistic models for subset selection are often built on a foundation of "repulsion." If you are picking a set of items—whether they are products for a recommendation carousel, images for a summary, or examples for a prompt—you generally want them to be diverse. Determinantal Point Processes (DPPs) became the standard tool for this because they elegantly model negative correlations: the presence of one item makes the presence of a similar item less likely.

However, real-world data is rarely just about diversity. In a grocery basket, a consumer might want diversity (not five types of bread), but they also want complementarity (bread and butter). Standard symmetric DPPs are mathematically incapable of modeling these "attractive" relationships. Recent research has focused on breaking this symmetry to capture the full spectrum of item interactions.

The Constraint of Symmetry

In a standard DPP, the probability of selecting a subset of items is proportional to the determinant of a submatrix of a symmetric kernel $L$. Because $L$ is symmetric and positive semidefinite, the determinant is geometrically related to the volume of a parallelepiped spanned by item feature vectors. This volume is maximized when vectors are orthogonal (diverse) and shrinks to zero when they are identical.

As noted by Mike Gartrell, Victor-Emmanuel Brunel, and others, this symmetry is a fundamental bottleneck. It forces all off-diagonal entries to represent negative associations. If you try to model a recommendation system where a phone and a charger frequently co-occur, a symmetric DPP simply cannot represent that positive correlation. This limitation extends to other models as well; for instance, Elise Epaillard points out that standard Dirichlet distributions are constrained by a rigid covariance structure that forces variables to be negatively correlated, necessitating more flexible "Generalized Dirichlet" mixtures.

Decomposing the Kernel

To capture both attraction and repulsion, researchers have turned to Nonsymmetric Determinantal Point Processes (NDPPs). The core intuition is to replace the symmetric kernel with a nonsymmetric one. A nonsymmetric matrix $L$ can be decomposed into two distinct parts:

$$L = S + A$$

  1. The Symmetric Part ($S$): This functions like a traditional DPP kernel, capturing the "repulsive" forces or the need for diversity.
  2. The Skew-Symmetric Part ($A$): Where $A_{ij} = -A_{ji}$. This component allows the model to capture "attractive" forces or complementarity.

By relaxing the symmetry constraint, the model can represent scenarios where the presence of one item actually increases the likelihood of another. Mike Gartrell and Syrine Krichene demonstrated that this decomposition allows for tractable maximum likelihood estimation, enabling models to learn these complex relationships directly from user behavior data.

Solving the Computational Hurdle

While nonsymmetric kernels are more expressive, they are also harder to work with. Finding the most likely subset (MAP inference) or sampling from the distribution becomes computationally expensive as the number of items grows.

To make these models practical, Aravind Reddy and Ryan A. Rossi developed "one-pass" streaming algorithms. Instead of looking at the entire dataset, these algorithms maintain a "stash" of promising candidates, allowing for real-time subset selection with minimal memory overhead.

For sampling, Insu Han and Amin Karbasi introduced scalable MCMC methods. By leveraging low-rank structures in the kernel, they reduced the complexity of the sampling step from quadratic to logarithmic ($O(\log n)$), making it possible to handle ground sets with millions of items.

Applications in Modern AI

The ability to balance internal consistency (attraction) with diversity (repulsion) has immediate benefits for Large Language Models (LLMs). In In-Context Learning (ICL), a model is given a few examples to follow. If the examples are too similar, the model doesn't generalize; if they are too random, they may be inconsistent.

Qing Du and Zhiliang Tian applied NDPPs to this problem, using the kernel decomposition to select example sets that are both diverse and mutually reinforcing. This "correlation-aware" selection ensures that the prompt is internally consistent while covering a wide range of the task space, leading to significant performance gains across various LLM benchmarks.

Beyond subset selection, these concepts are appearing in preference modeling. Siu Lun Chau and Dino Sejdinovic use skew-symmetric kernels within Gaussian Processes to model "inconsistent" preferences—cases where a user might prefer A over B and B over C, but C over A—further proving that breaking symmetry is essential for capturing the nuances of real-world data.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_modeling_attraction_and_repulsion_with_n_0346aaf9,
  title = {Modeling Attraction and Repulsion with Nonsymmetric Kernels},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/modeling-attraction-and-repulsion-with-nonsymmetric-kernels/rdr_0346aaf9fb2749bb947459c25fd4e605},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_modeling_attraction_and_repulsion_with_n_0346aaf9_v1,
  title = {Modeling Attraction and Repulsion with Nonsymmetric Kernels},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/modeling-attraction-and-repulsion-with-nonsymmetric-kernels/rdr_0346aaf9fb2749bb947459c25fd4e605/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}