Abstract
Diversification of recommendation results is a promising approach for coping with the uncertainty associated with users' information needs. Of particular importance in diversified recommendation is to define and optimize an appropriate diversity objective. In this study, we revisit the most popular diversity objective called intra-list distance (ILD), defined as the average pairwise distance between selected items, and a similar but lesser known objective called dispersion, which is the minimum pairwise distance. Our theoretical and experimental results reveal that these objectives have potential drawbacks: ILD may select duplicate items that are very close to each other, whereas dispersion may overlook distant item pairs. We propose Gaussian ILD (GILD), a diversity objective that interpolates between ILD and dispersion via a tunable bandwidth parameter, offering a more robust solution for recommendation diversity.
Introduction: The Diversity Dilemma
In modern recommendation systems, "diversity" is essential to prevent over-specialization and to account for the inherent uncertainty in user preferences. To achieve this, researchers typically optimize for specific diversity objectives during the ranking process.
The industry standard is Intra-List Distance (ILD), which calculates the average distance between all pairs of items in a list. A less common but theoretically interesting alternative is Dispersion, which focuses on the minimum distance between any two items—essentially a "worst-case" spread metric. Despite their widespread use, the specific item selection behaviors encouraged by these metrics have remained under-examined. This paper investigates whether maximizing these objectives truly results in a diverse list or if they harbor hidden biases.
Methodology
The authors employ a combination of theoretical proofs and empirical simulations to dissect these metrics.
Theoretical Analysis
Using approximation factors, the study quantifies the correlation between ILD and Dispersion. A key finding is an asymmetry in their relationship: while maximizing Dispersion provides a reasonable guarantee for ILD, the reverse is not true. Mathematically, maximizing ILD can lead to a Dispersion of zero, meaning the algorithm might select duplicate items if they are sufficiently far from other clusters.
Gaussian ILD (GILD)
To address the limitations of both metrics, the authors introduce Gaussian ILD (GILD). By applying a Gaussian kernel to pairwise distances, GILD creates a tunable spectrum:
- Large bandwidth ($\sigma$): GILD approximates the behavior of ILD, focusing on average spread.
- Small bandwidth ($\sigma$): GILD behaves like Dispersion, prioritizing minimum separation.
An "adjusted median" heuristic is proposed to automatically set $\sigma$, allowing the system to dynamically find a balance between global spread and local separation.
Key Insights: The "ILD Trap"
The most significant insight is the identification of the "ILD Trap." Because ILD averages distances, it can be maximized by selecting items from extreme, distant clusters while completely ignoring the similarity of items within those clusters.
In a practical scenario, a recommender might provide several nearly identical movies from one genre simply because they are very different from another set of movies in the list. Conversely, while Dispersion prevents such duplicates, it can be "myopic," potentially selecting a tight group of items that are just far enough apart to satisfy the minimum distance requirement without covering the full range of available content.
Experimental Results
The theoretical findings were validated using both synthetic and real-world datasets (MovieLens 1M and Amazon Magazines).
- Synthetic Tests: In the "Ellipse" test, ILD exclusively selected points from the extreme tips of the shape, whereas GILD (with adjusted $\sigma$) achieved a more uniform distribution across the entire cloud.
- Real-World Performance: On MovieLens, ILD-optimized lists frequently contained items with identical genre sets. When integrated with the EASE$^R$ recommendation model, GILD and Dispersion offered a superior trade-off between accuracy (nDCG) and diversity compared to ILD. Notably, ILD failed to improve minimum diversity metrics even as its trade-off parameter was increased.
Conclusion
This reexamination reveals that the popular ILD metric has a fundamental flaw: it is mathematically "lazy" regarding local item similarity. Gaussian ILD (GILD) emerges as a powerful alternative, providing a tunable framework that bridges the gap between average spread and minimum separation. Future work should focus on aligning these mathematical objectives with human perception of diversity through extensive user studies.
Summary updated