A Critical Reexamination of Intra-List Distance and Dispersion

Naoto Ohsaka, Riku Togashi

Abstract

Diversification of recommendation results is a promising approach for coping with the uncertainty associated with users' information needs. Of particular importance in diversified recommendation is to define and optimize an appropriate diversity objective. In this study, we revisit the most popular diversity objective called intra-list distance (ILD), defined as the average pairwise distance between selected items, and a similar but lesser known objective called dispersion, which is the minimum pairwise distance. Our theoretical and experimental results reveal that these objectives have potential drawbacks: ILD may select duplicate items that are very close to each other, whereas dispersion may overlook distant item pairs. We propose Gaussian ILD (GILD), a diversity objective that interpolates between ILD and dispersion via a tunable bandwidth parameter, offering a more robust solution for recommendation diversity.

Introduction: The Diversity Dilemma

In modern recommendation systems, "diversity" is essential to prevent over-specialization and to account for the inherent uncertainty in user preferences. To achieve this, researchers typically optimize for specific diversity objectives during the ranking process.

The industry standard is Intra-List Distance (ILD), which calculates the average distance between all pairs of items in a list. A less common but theoretically interesting alternative is Dispersion, which focuses on the minimum distance between any two items—essentially a "worst-case" spread metric. Despite their widespread use, the specific item selection behaviors encouraged by these metrics have remained under-examined. This paper investigates whether maximizing these objectives truly results in a diverse list or if they harbor hidden biases.

Methodology

The authors employ a combination of theoretical proofs and empirical simulations to dissect these metrics.

Theoretical Analysis

Using approximation factors, the study quantifies the correlation between ILD and Dispersion. A key finding is an asymmetry in their relationship: while maximizing Dispersion provides a reasonable guarantee for ILD, the reverse is not true. Mathematically, maximizing ILD can lead to a Dispersion of zero, meaning the algorithm might select duplicate items if they are sufficiently far from other clusters.

Gaussian ILD (GILD)

To address the limitations of both metrics, the authors introduce Gaussian ILD (GILD). By applying a Gaussian kernel to pairwise distances, GILD creates a tunable spectrum:

  • Large bandwidth ($\sigma$): GILD approximates the behavior of ILD, focusing on average spread.
  • Small bandwidth ($\sigma$): GILD behaves like Dispersion, prioritizing minimum separation.

An "adjusted median" heuristic is proposed to automatically set $\sigma$, allowing the system to dynamically find a balance between global spread and local separation.

Key Insights: The "ILD Trap"

The most significant insight is the identification of the "ILD Trap." Because ILD averages distances, it can be maximized by selecting items from extreme, distant clusters while completely ignoring the similarity of items within those clusters.

ILD's tendency to select items from extreme clusters, potentially ignoring internal similarity.

In a practical scenario, a recommender might provide several nearly identical movies from one genre simply because they are very different from another set of movies in the list. Conversely, while Dispersion prevents such duplicates, it can be "myopic," potentially selecting a tight group of items that are just far enough apart to satisfy the minimum distance requirement without covering the full range of available content.

Comparison of ILD and Dispersion selection patterns in worst-case scenarios.

Experimental Results

The theoretical findings were validated using both synthetic and real-world datasets (MovieLens 1M and Amazon Magazines).

  • Synthetic Tests: In the "Ellipse" test, ILD exclusively selected points from the extreme tips of the shape, whereas GILD (with adjusted $\sigma$) achieved a more uniform distribution across the entire cloud.
  • Real-World Performance: On MovieLens, ILD-optimized lists frequently contained items with identical genre sets. When integrated with the EASE$^R$ recommendation model, GILD and Dispersion offered a superior trade-off between accuracy (nDCG) and diversity compared to ILD. Notably, ILD failed to improve minimum diversity metrics even as its trade-off parameter was increased.

Conclusion

This reexamination reveals that the popular ILD metric has a fundamental flaw: it is mathematically "lazy" regarding local item similarity. Gaussian ILD (GILD) emerges as a powerful alternative, providing a tunable framework that bridges the gap between average spread and minimum separation. Future work should focus on aligning these mathematical objectives with human perception of diversity through extensive user studies.

Summary updated

Abstract

Diversification of recommendation results is a promising approach for coping with the uncertainty associated with users' information needs. Of particular importance in diversified recommendation is to define and optimize an appropriate diversity objective. In this study, we revisit the most popular diversity objective called intra-list distance (ILD), defined as the average pairwise distance between selected items, and a similar but lesser known objective called dispersion, which is the minimum pairwise distance. Owing to their simplicity and flexibility, ILD and dispersion have been used in a plethora of diversified recommendation research. Nevertheless, we do not actually know what kind of items are preferred by them. We present a critical reexamination of ILD and dispersion from theoretical and experimental perspectives. Our theoretical results reveal that these objectives have potential drawbacks: ILD may select duplicate items that are very close to each other, whereas dispersion may overlook distant item pairs. As a competitor to ILD and dispersion, we design a diversity objective called Gaussian ILD, which can interpolate between ILD and dispersion by tuning the bandwidth parameter. We verify our theoretical results by experimental results using real-world data and confirm the extreme behavior of ILD and dispersion in practice.
BibTeX
@inproceedings{ohsaka2023critical,
  author = {Naoto Ohsaka and Riku Togashi},
  title = {{A Critical Reexamination of Intra-List Distance and Dispersion}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2023},
  doi = {10.1145/3539618.3591623},
}
Figure