Abstract
DeepSRGM introduces a deep learning framework that treats Indian Classical Music (ICM) as a sequence modeling problem, analogous to Natural Language Processing. By utilizing LSTMs with attention mechanisms and triplet loss, the researchers achieved state-of-the-art Raga recognition accuracy (up to 97.1%) and enabled a new "sequence ranking" feature for retrieving similar melodic patterns from large databases. This approach effectively captures the temporal grammar and characteristic motifs that define the melodic identity of a Raga.
Introduction
Identifying a Raga—the melodic framework of Indian Classical Music—is a notoriously difficult task for machines. Unlike Western scales, a Raga is not merely a set of notes; it is defined by its temporal structure, specific ornaments (gamakas), and rules for ascending (arohana) and descending (avarohana). Two Ragas might share the exact same notes but sound completely different based on which notes are emphasized or how a singer transitions between them.
Previous automated approaches often relied on Pitch Class Distributions (PCD), which act like a "bag of notes," ignoring vital temporal order. Other methods relied on handcrafted features that struggled to scale. DeepSRGM aims to capture the "grammar" of ICM by modeling the temporal sequences of notes directly using Recurrent Neural Networks (RNNs).
Methods
The authors propose two models: SRGM1 for classification and SRGM2 for sequence ranking. The pipeline involves several sophisticated preprocessing steps to prepare the raw audio for the neural network:
- Source Separation: Using Mad-TwinNet to isolate the lead vocal from accompanying instruments.
- Pitch Tracking & Normalization: Extracting the fundamental frequency and normalizing it relative to the artist's "Tonic" (base note), then quantizing the pitch into discrete levels.
- SRGM1 (The Classifier): An LSTM-RNN architecture with an Attention Layer. The model treats sequences of pitch values as "sentences" and learns to identify which Raga they belong to.
- SRGM2 (The Ranker): The authors fine-tune the classifier using Triplet Margin Loss. This forces the model to create "feature embeddings" where melodic phrases from the same Raga are mathematically close to each other, while phrases from different Ragas are pushed apart.
- Inference via Voting: To classify a full song, the model takes multiple random subsequences, predicts the Raga for each, and uses a majority voting system to produce the final label.
Key Results and Insights
The study utilized the Comp Music Carnatic Dataset (CMD), featuring 124 hours of high-quality live concert recordings across 40 different Ragas. The results demonstrate that treating music as a language allows the model to learn the characteristic melodic phrases (pakads) that define a Raga.
- State-of-the-Art Accuracy: On the 10-Raga subset, the ensemble model achieved 97.1% accuracy, outperforming previous methods like TDMS and PCD-based models. On the much harder 40-Raga dataset, the model achieved 88.1% accuracy.
- Sequence Length Impact: The researchers demonstrated that longer training sequences (6,000 steps) converge significantly faster and achieve higher accuracy than shorter ones, proving that the "context" of the melody is crucial.
- Retrieval Performance: The SRGM2 model achieved a top-10 precision of 81.68% for melodic sequence retrieval, providing a vital tool for musicologists to find similar phrases across massive databases.
Limitations and Future Work
Currently, the model struggles with audio segments containing long pauses or sustained single notes, as these provide very little melodic information for the LSTM to process. Additionally, the system requires the "Tonic" (the base reference pitch) to be known. Future work aims to build a model that can jointly estimate the tonic and the Raga simultaneously. The authors also suggest that this sequence modeling approach could eventually be applied to generative models capable of replicating the intricate, improvised aspects of Indian Classical Music.
Summary updated