Towards an Integrated Approach to Music Retrieval

Emanuele Di Buccio, Ivano Masiero, Yosi Mass, Massimo Melucci, Riccardo Miotto, Nicola Orio, Benjamin SznajderIRCDL 2009 · OpenReview

Abstract

This paper presents a hybrid music retrieval system designed for large-scale peer-to-peer (P2P) networks. By combining content-based melodic features with textual metadata, the system aims to bridge the "semantic gap" in music discovery. The architecture integrates two search modalities—textual and melodic—into a single interface, allowing for complex queries that combine sound similarity with specific attributes like artist, genre, or tempo.

Introduction

Most music search in P2P networks relies on simple file name matching or metadata that is often incomplete, inconsistent, or incorrect. While content-based retrieval—finding music based on its actual sound or melody—is a powerful alternative, it faces two major hurdles: the "semantic gap," where raw audio features do not map neatly to human musical concepts, and scalability, as content-based indexing is computationally expensive in decentralized environments. This research addresses these challenges within the context of the SAPIR project, proposing an architecture that integrates multiple search modalities into a unified framework.

Methodology

The system processes music documents, specifically MIDI files, through two parallel tracks to ensure comprehensive indexing and retrieval.

Multi-Modal Description

The system employs a two-pronged approach to document description:

  1. Textual Metadata: Metadata is extracted directly from MIDI files. To overcome the unreliability of internal tags, a "focused crawler" analyzes the web pages where the files were originally hosted, associating reliable artist and title information with the documents based on their web context.
  2. Content-based Descriptors: The system identifies the "leading voice" or main melody of a track. It extracts two key features: Pitch Intervals (PIT) and Interonset Interval log-ratios (IOI), which represent rhythmic timing. These features are quantized into a "musical alphabet," effectively transforming a melody into a string of symbols that can be indexed using standard information retrieval techniques.

Hybrid P2P Architecture

The system utilizes two distinct overlay networks to manage different data types efficiently:

  • A text overlay (developed by IBM) handles metadata-based queries.
  • A music content overlay (called SPINA) manages melodic similarity searches.

Integrated Search and Data Fusion

When a user performs a "Query By Example" by uploading a MIDI file and applying text filters, the system queries both overlays simultaneously. It then employs data fusion and reranking strategies to combine the results into a single, coherent list. This allows users to find songs that "sound like" a provided example while meeting specific metadata criteria, such as a "fast" tempo or a specific artist.

Evaluation and Results

The researchers compiled a dataset of approximately 60,000 MIDI files through web crawling to test the system's performance.

Metadata Accuracy

The focused crawling approach proved highly effective, providing reliable labels for 90% of the dataset. This demonstrates that external web context is a superior source of "ground truth" compared to internal file metadata, which is often missing or erroneous.

Retrieval Performance

In centralized testing on a controlled set of 600 documents, the system achieved a Mean Average Precision (MAP) of 0.54. This is a respectable result for content-based music retrieval in a noisy, real-world dataset. Furthermore, the architecture demonstrated the ability to handle tens of thousands of files across a simulated P2P network with realistic response times, proving its scalability.

Discussion

A key innovation of this work is the treatment of quantized melodic patterns as "words," allowing for the application of standard techniques like TF-IDF weighting to music. This significantly improves search efficiency compared to raw waveform comparison. Additionally, the use of specialized overlays for different media types enhances the system's scalability and robustness in dynamic P2P environments where nodes frequently join and leave.

Conclusion and Future Work

The hybrid approach successfully bridges the semantic gap and provides a scalable solution for music retrieval in decentralized networks. Future work will focus on extending these methods to raw audio formats like MP3, which present greater challenges due to polyphony and background noise. Additionally, the researchers plan to investigate the system's resilience to P2P "churn" and conduct user studies to optimize the "Query By Example" paradigm for non-expert users.

Summary updated

BibTeX
@inproceedings{buccio2009towards,
  author = {Emanuele Di Buccio and Ivano Masiero and Yosi Mass and Massimo Melucci and Riccardo Miotto and Nicola Orio and Benjamin Sznajder},
  title = {{Towards an Integrated Approach to Music Retrieval}},
  booktitle = {IRCDL},
  year = {2009},
  url = {https://openreview.net/forum?id=mqIT6ND3wuT},
}
Figure