Abstract
The long-tail phenomenon tells us that there are many items in the tail. However, not all tail items are the same. Each item acquires different kinds of users. Some items are loved by the general public, while some items are consumed by eccentric fans. In this paper, we propose a novel metric, item eccentricity, to incorporate this difference between consumers of the items. Eccentric items are defined as items that are consumed by eccentric users. We used this metric to analyze two real-world datasets of music and movies and observed the characteristics of items in terms of eccentricity. The results showed that our defined eccentricity of an item does not change much over time, and classified eccentric and noneccentric items present significantly distinct characteristics. The proposed metric effectively separates the eccentric and noneccentric items mixed in the tail, which could not be done with the previous measures, which only consider the popularity of items.
Introduction: Beyond the Long Tail
In modern recommender systems, user-item interactions typically follow a power-law distribution. A small number of "head" items receive the majority of interactions, while a vast "long tail" of items remains largely unpopular. Traditional recommendation algorithms often suffer from popularity bias, favoring head items to maximize aggregate accuracy. While recent research has focused on surfacing tail items to improve serendipity, these approaches often treat all rare items as a homogeneous group.
This paper argues that not all unpopular items are created equal. Some items are obscure due to being new or of low quality, while others are "eccentric"—they appeal specifically to a niche audience with unconventional tastes. By introducing the metric of item eccentricity, we can differentiate between these types of tail items, offering a more nuanced understanding of user preferences.
Methodology: Quantifying Eccentricity
The authors propose a recursive mathematical framework to quantify eccentricity, calculated over rolling four-week windows to account for temporal shifts in popularity:
- Item Rarity: This baseline metric is the negative log of the number of feedbacks an item receives. It captures the standard notion of unpopularity.
- User Eccentricity: This measures a user's long-term tendency to consume rare items. It is the average rarity of items consumed by a user, weighted by their feedback (e.g., play counts or ratings).
- Item Eccentricity: The core metric, defined as the average User Eccentricity of the individuals who interacted with the item.
Crucially, an item is not deemed eccentric simply because it is rare; it is eccentric because it attracts users who consistently seek out niche content across the platform.
Distributional Characteristics
One of the most significant findings is the distributional difference between rarity and eccentricity. While rarity follows a heavy long-tailed distribution, item eccentricity follows a bell-curved (normal) distribution.
As shown in the figure above, both user and item eccentricity across the 30Music and MovieLens datasets exhibit a centralized, normal distribution. This suggests that while the majority of items in a system are rare, only a specific subset can be classified as truly eccentric. This distinction allows system designers to separate "niche" items from "unpopular" items.
Temporal Stability of Eccentricity
For a metric to be useful in a recommendation context, it must represent a stable characteristic of the item rather than statistical noise. The authors tested this by splitting the datasets into two distinct time periods and calculating the probability of an item remaining in the same eccentricity "septile" (one-seventh of the distribution).
The transition matrices reveal remarkable stability. In the MovieLens dataset, items in the highest eccentricity septile had an 84% probability of remaining there in the subsequent period. In the 30Music dataset, the retention was 71%. This high diagonal density confirms that eccentricity is an inherent, persistent trait of items.
Engagement and Polarization
The researchers further analyzed how eccentric items behave in terms of user engagement:
- Music (30Music): Eccentric tracks inspire deeper loyalty. When comparing items with identical rarity scores, eccentric tracks had significantly higher replay rates and were more frequently "loved" by users compared to non-eccentric rare tracks.
- Movies (MovieLens): Eccentric movies tend to be polarizing. They often exhibit a multimodal rating distribution, meaning they are either loved or hated, whereas non-eccentric rare movies tend to have more centralized, mediocre ratings.
Conclusion and Future Directions
Item eccentricity provides a powerful lens for analyzing the long tail. By identifying items that appeal to niche, loyal audiences, recommender systems can move beyond simple popularity-based metrics. While the current work focuses on exploratory analysis and faces challenges like the "cold start" problem for new items, the stability and distinct characteristics of eccentric items suggest they are prime candidates for improving serendipitous recommendations. Future work should focus on integrating eccentricity directly into ranking algorithms to better serve users with unconventional tastes.
Summary updated