Introduction
Named Entity Recognition (NER) is a fundamental task in Natural Language Processing (NLP), yet most existing systems are optimized for single-sentence processing. This paper investigates the "context gap" in Marathi NER, proposing a method to extend recognition capabilities to long-range document contexts using the MahaBERT transformer model.
The Challenge of Long-Range Context
Marathi is a morphologically rich language with diverse writing styles. Traditional NER systems often fail to capture entities or relationships that span across sentence boundaries. This limitation is a significant hurdle for document-level applications such as automated knowledge extraction and sentiment analysis within the Marathi digital ecosystem.
Methodology
The researchers utilized MahaBERT, a BERT variant pre-trained specifically for Marathi. To address the lack of long-form annotated data, they employed contextual data augmentation. By concatenating existing annotated sentences from the MahaNER dataset, they created synthetic "long-range" samples. The model was fine-tuned using specific hyperparameters, including the Adam optimizer, to handle these extended sequences.
Datasets and Data Augmentation
The study centers on the MahaNER dataset, comprising 25,000 manually annotated sentences. To simulate document-level text, four derivative datasets were curated: Concat2 (two sentences), Concat3 (three sentences), Concat-similar (four sentences with topical repetition), and a Combined dataset containing a mixture of all lengths.
As shown in the figure above, the concatenation process significantly increases the sequence length, providing the necessary "long-range" context for training.
Key Results and Performance
The experiments demonstrated that models trained only on short sentences suffer a significant performance drop when faced with longer text. Specifically, the baseline model's F1-score fell from 84.19% to 72.64% when tested on long-range sequences.
However, training on the "Combined" dataset allowed the model to recover and even exceed baseline performance on long-range text, achieving an 85.46% F1-score. Crucially, learning long-range dependencies did not degrade the model's performance on standard single-sentence tasks, maintaining high F1-scores across all configurations.
Conclusion
This research proves that synthetic context augmentation is a viable and effective strategy for improving long-range NER in Marathi. By exposing models to varied sequence lengths, researchers can build more robust systems capable of handling complex, document-level information without the need for expensive new human-annotated datasets. Future work will explore specialized domains like legal text and more efficient model architectures.
Summary updated