Introduction
Sentiment analysis is a cornerstone of Natural Language Processing (NLP), yet its efficacy often diminishes when applied to specialized domains such as law. This performance degradation is primarily attributed to negative transfer, where models trained on general-purpose datasets—such as movie reviews—fail to account for the "emotional weight" shifts of specific terminology in a legal context. For instance, the word "charged" may denote excitement in a cinematic review but carries a strictly negative connotation in a courtroom setting.
Furthermore, the legal domain represents a "low-resource setting." The high cost of expert legal annotation makes the creation of large-scale labeled datasets prohibitively expensive. This research proposes a semi-automated framework to bridge this gap by identifying "sentiment-shifting" words, enabling targeted manual annotation with minimal human effort.
Methodology
The proposed approach adapts general-purpose sentiment models to the legal domain through a multi-stage pipeline:
- Word Categorization: Vocabulary is divided into Domain Generic (consistent meaning), Domain Specific (shifting meaning), and Under Represented (rare in general text but frequent in legal documents).
- Heuristic Identification: To detect sentiment shifts without exhaustive manual labeling, the researchers utilized Word2Vec embeddings trained on movie reviews (IMDB) and legal texts (SigmaLaw). A
domainSimilarityscore, based on cosine similarity between the most similar words in each domain, was used to flag words exhibiting divergent behavior. - Targeted Annotation: Legal experts annotated only the flagged words and a small subset of sentences (925 words and 576 sentences), significantly reducing the manual workload.
- Model Adaptation:
- RNTN (Recursive Neural Tensor Network): Fine-tuned by replacing vectors of "deviated" words with vectors of words possessing the correct sentiment in the source domain.
- BERT: The training set was filtered to remove conflicting sentences, and the newly annotated legal data was integrated to create $BERT_l$.
Key Results
The effectiveness of the domain adaptation was evaluated by comparing the improved models ($RNTN_l$ and $BERT_l$) against baselines trained solely on source-domain data ($RNTN_m$ and $BERT_m$).
As shown in the performance data below, the results demonstrate a significant performance leap. The baseline $BERT_m$ achieved 57% accuracy on legal texts, while the adapted $BERT_l$ reached 67% accuracy—a 10% improvement achieved with fewer than 600 new annotations. Similarly, the $RNTN_l$ model saw an accuracy increase from 48% to 57%.
Beyond overall accuracy, the heuristic approach demonstrated high precision in identifying sentiment-shifting words. The system identified negative sentiment words with 80% precision, representing a substantial improvement over the baseline's internal assignments. The $RNTN_l$ model also showed much more consistent performance across all three classes (Positive, Neutral, Negative) compared to the movie-only models, which often struggled with the positive class in a legal context.
Discussion and Future Work
A critical insight from this study is that "Neutrality is a trap." Many words that carry sentiment in general usage become neutral in a legal context, and vice versa. Managing the "Neutral" class remains a challenge due to its sheer volume (nearly 5,000 words in the source set).
While the current study focuses on a 3-class sentiment problem (Positive, Neutral, Negative), future research could extend this heuristic approach to more granular scales (e.g., 5-class) or other low-resource domains like medicine or engineering, where specialized terminology similarly drifts from standard English usage.
Summary updated