Abstract
The version identification (VI) task deals with the automatic detection of recordings that correspond to the same underlying musical piece. Despite many efforts, VI is still an open problem, with much room for improvement, specially with regard to combining accuracy and scalability. In this paper, we present MOVE, a musically-motivated method for accurate and scalable version identification. MOVE achieves state-of-the-art performance on two publicly-available benchmark sets by learning scalable embeddings in an Euclidean distance space, using a triplet loss and a hard triplet mining strategy. It improves over previous work by employing an alternative input representation, and introducing a novel technique for temporal content summarization, a standardized latent space, and a data augmentation strategy specifically designed for VI. In addition to the main results, we perform an ablation study to highlight the importance of our design choices, and study the relation between embedding dimensionality and model performance.
Introduction
Version Identification (VI) is the task of automatically detecting if two recordings represent the same underlying musical piece. Unlike simple audio fingerprinting, which looks for exact digital duplicates, VI must handle significant changes in tempo, key, instrumentation, and even song structure. While traditional methods using local alignment (like Dynamic Time Warping) are accurate, they are computationally expensive and do not scale to datasets with millions of songs. Recent deep learning approaches have attempted to learn low-dimensional embeddings, but many struggle to balance accuracy with the need for a standardized, searchable latent space.
MOVE (Musically-motivated Version Embeddings) addresses these challenges by mapping musical recordings into a searchable Euclidean space. By combining a novel input representation with musically-informed architecture choices, it achieves state-of-the-art accuracy while remaining scalable enough for industrial-scale applications.
Methods
The MOVE framework is a convolutional neural network (CNN) designed with several musically-motivated components, as illustrated in Figure 1.
Key architectural choices include:
- Crema-PCP Input: Instead of standard chromagrams, MOVE uses "crema-PCP," a representation derived from a pre-trained chord estimation model. This captures harmonic information more robustly than raw spectral features.
- Transposition Invariance: To handle key changes, the model uses a "circular" convolution strategy. It doubles the input pitch bins (from 12 to 23) and uses a specific kernel size followed by max-pooling to ensure the network identifies the same song regardless of the key it's played in.
- Multi-channel Adaptive Attention: To summarize a whole song into a single vector, MOVE uses a hybrid of temporal attention and "auto-pool." This allows the network to learn which parts of a song are most representative and dynamically pivot between average-pooling and max-pooling.
- Standardized Latent Space: The final layer uses non-parametric batch normalization. This forces the embeddings into a zero-mean, unit-variance distribution, making the Euclidean distances more meaningful and easier to optimize during training.
- Training Strategy: The model is trained using a triplet loss with an online "hard triplet mining" strategy. The authors found that selecting "moderate" hard triplets—rather than the absolute hardest—leads to better convergence.
Results and Discussion
MOVE was evaluated on two major benchmarks: Da-TACOS (15,000 songs) and YouTubeCovers (YTC) (50 cliques with 7 versions each). As shown in the results table below, MOVE significantly outperforms previous state-of-the-art (SOTA) systems.
On Da-TACOS, MOVE achieved a Mean Average Precision (MAP) of 0.507, a significant jump over the previous best of 0.454 (LateFusion). It also drastically improved the Mean Rank (MR1) from 177 down to 40. On YTC, MOVE reached a MAP of 0.889, surpassing the previous SOTA (CQT-TPPNet at 0.859).
Ablation studies highlighted the critical nature of these design choices. Removing the transposition-invariant architecture caused the biggest performance hit, with MAP dropping from 0.575 to 0.154. Additionally, the authors studied the impact of embedding dimensionality ($d$), finding that performance improves as $d$ increases but hits a point of diminishing returns around $d=16k$. Notably, even a smaller $4k$ embedding outperforms previous SOTA models.
Conclusion
MOVE demonstrates that musically-informed design—specifically regarding transposition invariance and temporal summarization—is essential for effective version identification. By producing Euclidean embeddings, MOVE is compatible with high-speed nearest-neighbor libraries, making it viable for real-world digital rights management. Future work will explore incorporating additional musical dimensions, such as melody or rhythm, and improving the interpretability of the learned similarity concepts.
Summary updated