DeepSRGM - Sequence Classification and Ranking in Indian Classical Music with Deep Learning

Sathwik Tejaswi Madhusudhan, Girish ChowdharyarXiv 2024 · DOI · arXiv

Abstract

DeepSRGM introduces a deep learning framework that treats Indian Classical Music (ICM) as a sequence modeling problem, analogous to Natural Language Processing. By utilizing LSTMs with attention mechanisms and triplet loss, the researchers achieved state-of-the-art Raga recognition accuracy (up to 97.1%) and enabled a new "sequence ranking" feature for retrieving similar melodic patterns from large databases. This approach effectively captures the temporal grammar and characteristic motifs that define the melodic identity of a Raga.

Introduction

Identifying a Raga—the melodic framework of Indian Classical Music—is a notoriously difficult task for machines. Unlike Western scales, a Raga is not merely a set of notes; it is defined by its temporal structure, specific ornaments (gamakas), and rules for ascending (arohana) and descending (avarohana). Two Ragas might share the exact same notes but sound completely different based on which notes are emphasized or how a singer transitions between them.

Previous automated approaches often relied on Pitch Class Distributions (PCD), which act like a "bag of notes," ignoring vital temporal order. Other methods relied on handcrafted features that struggled to scale. DeepSRGM aims to capture the "grammar" of ICM by modeling the temporal sequences of notes directly using Recurrent Neural Networks (RNNs).

Methods

The authors propose two models: SRGM1 for classification and SRGM2 for sequence ranking. The pipeline involves several sophisticated preprocessing steps to prepare the raw audio for the neural network:

  1. Source Separation: Using Mad-TwinNet to isolate the lead vocal from accompanying instruments.
  2. Pitch Tracking & Normalization: Extracting the fundamental frequency and normalizing it relative to the artist's "Tonic" (base note), then quantizing the pitch into discrete levels.
  3. SRGM1 (The Classifier): An LSTM-RNN architecture with an Attention Layer. The model treats sequences of pitch values as "sentences" and learns to identify which Raga they belong to.
  4. SRGM2 (The Ranker): The authors fine-tune the classifier using Triplet Margin Loss. This forces the model to create "feature embeddings" where melodic phrases from the same Raga are mathematically close to each other, while phrases from different Ragas are pushed apart.
  5. Inference via Voting: To classify a full song, the model takes multiple random subsequences, predicts the Raga for each, and uses a majority voting system to produce the final label.
Overview of the DeepSRGM preprocessing and classification pipeline.

Key Results and Insights

The study utilized the Comp Music Carnatic Dataset (CMD), featuring 124 hours of high-quality live concert recordings across 40 different Ragas. The results demonstrate that treating music as a language allows the model to learn the characteristic melodic phrases (pakads) that define a Raga.

  • State-of-the-Art Accuracy: On the 10-Raga subset, the ensemble model achieved 97.1% accuracy, outperforming previous methods like TDMS and PCD-based models. On the much harder 40-Raga dataset, the model achieved 88.1% accuracy.
  • Sequence Length Impact: The researchers demonstrated that longer training sequences (6,000 steps) converge significantly faster and achieve higher accuracy than shorter ones, proving that the "context" of the melody is crucial.
  • Retrieval Performance: The SRGM2 model achieved a top-10 precision of 81.68% for melodic sequence retrieval, providing a vital tool for musicologists to find similar phrases across massive databases.
Comparative performance of SRGM1 against baseline methods on the CMD-10 and CMD-40 datasets.

Limitations and Future Work

Currently, the model struggles with audio segments containing long pauses or sustained single notes, as these provide very little melodic information for the LSTM to process. Additionally, the system requires the "Tonic" (the base reference pitch) to be known. Future work aims to build a model that can jointly estimate the tonic and the Raga simultaneously. The authors also suggest that this sequence modeling approach could eventually be applied to generative models capable of replicating the intricate, improvised aspects of Indian Classical Music.

Summary updated

Abstract

A vital aspect of Indian Classical Music (ICM) is Raga, which serves as a melodic framework for compositions and improvisations alike. Raga Recognition is an important music information retrieval task in ICM as it can aid numerous downstream applications ranging from music recommendations to organizing huge music collections. In this work, we propose a deep learning based approach to Raga recognition. Our approach employs efficient pre possessing and learns temporal sequences in music data using Long Short Term Memory based Recurrent Neural Networks (LSTM-RNN). We train and test the network on smaller sequences sampled from the original audio while the final inference is performed on the audio as a whole. Our method achieves an accuracy of 88.1% and 97 % during inference on the Comp Music Carnatic dataset and its 10 Raga subset respectively making it the state-of-the-art for the Raga recognition task. Our approach also enables sequence ranking which aids us in retrieving melodic patterns from a given music data base that are closely related to the presented query sequence.
BibTeX
@misc{madhusudhan2024deepsrgm,
  author = {Sathwik Tejaswi Madhusudhan and Girish Chowdhary},
  title = {{DeepSRGM - Sequence Classification and Ranking in Indian Classical Music with Deep Learning}},
  year = {2024},
  eprint = {2402.10168},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.10168},
}
Figure