Abstract
Recommender Systems are an integral part of music sharing platforms. Often the aim of these systems is to increase the time, the user spends on the platform and hence having a high commercial value. The systems which aim at increasing the average time a user spends on the platform often need to recommend songs which the user might want to listen to next at each point in time. This is different from recommendation systems which try to predict the item which might be of interest to the user at some point in the user lifetime but not necessarily in the very near future. Prediction of the next song the user might like requires some kind of modeling of the user interests at the given point of time. Attentive neural networks have been exploiting the sequence in which the items were selected by the user to model the implicit short-term interests of the user for the task of next item prediction, however we feel that the features of the songs occurring in the sequence could also convey some important information about the short-term user interest which only the items cannot. In this direction, we propose a novel attentive neural architecture which in addition to the sequence of items selected by the user, uses the features of these items to better learn the user short-term preferences and recommend the next song to the user.
Introduction
Most music recommendation systems fall into two camps: those that suggest what you might like eventually (long-term preference) and those that suggest what you want to hear right now (short-term interest). The latter is significantly more challenging because user moods and contexts shift rapidly.
While Recurrent Neural Networks (RNNs) are effective at modeling sequences, they often struggle to capture the nuance of why a user is listening to a specific set of songs. Standard sequence models often assume the most recent item is the most important. However, in music, a song played several tracks ago might be a better indicator of a user's current mood than the very last track played, which might have been an accidental play or a skip. This research aims to bridge this gap by incorporating song features (tags) and an attention mechanism that can selectively prioritize relevant past interactions.
Methods
The proposed STABR (Song and Tag Attention Based Recommendation) architecture consists of two parallel attentive components that merge to produce a final recommendation.
- The Song Component: This path processes the sequence of song IDs. These IDs are converted into embeddings and passed through a Bidirectional Gated Recurrent Unit (Bi-GRU). An attention layer then assigns weights to these hidden states, allowing the model to focus on specific songs in the history regardless of their temporal position.
- The Tag Component: Instead of relying solely on song IDs, this path incorporates metadata. For every song in the history, the model takes the average embedding of all its associated Last.fm tags (e.g., "rock," "mellow," "80s"). These are also processed via a Bi-GRU and an attention layer.
- Fusion and Prediction: The "context vectors" from both paths are concatenated and passed through a bottleneck dense layer to reduce dimensionality. Finally, a softmax layer calculates the probability distribution for the next song across the entire catalog.
Dataset and Experimental Setup
The study utilizes a subset of the Last.fm dataset to evaluate the model. The dataset includes over 3.5 million logs across 759 users and 110,410 sessions, featuring approximately 386,046 unique songs and 487,844 unique tags.
For preprocessing, user histories were divided into sessions. The first 70% of sessions were used for training, and the remaining 30% for testing. Sessions with fewer than five songs were discarded to ensure sufficient context for the attention mechanism to function effectively.
Key Results
The models were evaluated using HitRatio@k, which measures the probability that the ground-truth next song appears within the top k recommendations.
As shown in the results, STABR significantly outperforms all baselines. At $k=10$, STABR achieved a HitRatio of 28.95%, more than doubling the performance of the standard RNN baseline (14.42%). It also outperformed the song-only attention model (SABR), proving that the inclusion of tag-based metadata is essential for capturing short-term user preferences.
Discussion and Conclusion
The success of STABR highlights two critical insights: first, that attention mechanisms are superior to simple recency-based models in music recommendation; and second, that tags provide a dense semantic layer that helps bridge the sparsity of song IDs. By averaging tag embeddings, the model can recommend songs based on their "profile" even if they haven't appeared in a specific sequence before.
Future work may address the computational complexity of performing a softmax over hundreds of thousands of songs, potentially through negative sampling or hierarchical softmax. Additionally, exploring the model's robustness in environments with noisier or sparser metadata remains an area for further investigation.
Summary updated