Introduction
Polarity lexicons, which map words to sentiments like positive, negative, or neutral, are fundamental to sentiment analysis. However, general-purpose lexicons often struggle with domain-specific nuances. For instance, the word "unpredictable" might be a positive attribute for a thriller's plot but a negative one for a vehicle's handling. This paper presents a method to automatically adapt an existing lexicon to a target domain using Integer Linear Programming (ILP).
The Challenge of Domain-Specific Sentiment
Standard lexicons are typically built in isolation or derived from general corpora. When applied to specific domains, they often fail to capture how word meanings shift. While machine learning models can learn sentiment directly from data, they often ignore the valuable prior knowledge contained in curated dictionaries. The goal of this research is to bridge this gap by adapting general-purpose lexicons to reflect the specific lexical usage of a target domain.
Methodology: Lexicon Adaptation via ILP
The proposed approach frames lexicon adaptation as an optimization problem. Word-level polarities are treated as latent variables, while known expression-level polarities from a domain corpus serve as guidance.
Bipartite Graph Representation
The system represents the relationship between words and the expressions they inhabit as a bipartite graph. This allows the model to consider both intra-expression relations (how words interact within a single phrase) and inter-expression relations (how a word's behavior varies across different contexts).
Optimization and Constraints
The ILP formulation assigns binary variables to potential word polarities (positive, neutral, negative, or negator). To ensure linguistic validity, the model enforces constraints, such as limiting the number of "content-word negators" (e.g., "prevent", "destroy") per expression to avoid overly complex double-negatives.
A feedback loop involving a sentiment classifier is used to refine the lexicon. If a classifier misidentifies an expression's sentiment using the default lexicon, the ILP is penalized unless it adjusts word-level polarities to correct the error. Conversely, correct predictions encourage the ILP to maintain the original lexicon's assignments. To maintain computational tractability, the search space is restricted to an "edit-one distance," meaning the ILP only considers changing one word's polarity at a time for a given expression.
Experimental Results
The method was evaluated using the MPQA (Multi-Perspective Question Answering) corpus and a general-purpose lexicon augmented with the General Inquirer dictionary.
The results highlight two key findings. First, incorporating lexicon features into a standard Conditional Random Field (CRF) classifier significantly boosts performance over a purely data-driven approach. Second, the adaptation process further refines these gains.
As shown in the results, using the adapted lexicon with a CRF classifier improved accuracy from 70.4% to 71.2% and reduced the average error distance. While the numerical gain might seem modest, it represents a statistically significant improvement over a strong baseline, demonstrating that domain-specific lexical tuning provides a tangible edge.
Conclusion
This research demonstrates that lexicons remain vital in the era of machine learning. By balancing "competence" (prior dictionary knowledge) with "performance" (empirical data distribution), the ILP-based adaptation method creates more accurate, context-aware lexical resources. Future work will explore moving beyond static dictionaries toward even more dynamic, sense-aware adaptation techniques.
Summary updated