Introduction
Monolingual alignment is the fundamental task of identifying semantic correspondences between words or phrases in two sentences within the same language. This process is critical for various high-level natural language processing (NLP) applications, including paraphrase detection, question answering, and the evaluation of machine translation systems. Unlike bilingual alignment, which often relies on large parallel corpora, monolingual alignment requires a deep understanding of context and semantic similarity to resolve ambiguities.
Problem Definition
The primary challenge in monolingual alignment lies in balancing local lexical similarity with global sentence structure. Words do not exist in isolation; they "compete" or "cooperate" for alignments. For instance, in the sentences "UK troops stormed a police station" and "British armor crashed into a jail," the word "troops" might locally match with "soldiers" elsewhere in the text, making it a better fit than "armor." Previous alignment models often struggled to integrate these interdependencies, leading to errors where local matches were prioritized over globally coherent structures.
Methodology: Two-Stage Logistic Regression
To address these challenges, we propose a feature-rich, two-stage logistic regression framework that separates local evidence gathering from global decision-making.
Stage 1: Local Assessment
The first stage calculates an alignment probability ($\phi_{ij}$) for every possible word pair $(i, j)$ based on local features. These features include:
- Lexical Similarity: Utilizing neural word embeddings and the Paraphrase Database (PPDB) via a ridge regression model.
- Contextual Features: Incorporating both dependency-based (syntax tree relations) and surface-form (neighboring words) similarities.
- Word Typing: Training specialized models for different categories of word pairs (e.g., content words vs. function words) to capture category-specific alignment patterns.
Stage 2: Global Refinement
The second stage assigns final alignment labels by refining the Stage 1 probabilities through a global lens. This stage introduces features that account for:
- Competition: Evaluating if a word has a stronger potential match elsewhere in the sentence.
- Cooperation: Identifying if neighboring words provide mutual support for an alignment (e.g., components of a named entity).
- Optimal Matching: Employing a max-weighted bipartite matching algorithm to determine the mathematically optimal set of one-to-one pairings across the entire sentence pair.
Evaluation and Results
The model was evaluated on the MSR Alignment Corpus and the Edinburgh++ Corpus. It achieved state-of-the-art results, reaching an $F_1$ score of 92.1% on the MSR corpus (a 4.8% error reduction) and 88.5% on Edinburgh++ (a 7.3% error reduction).
Extrinsic evaluation on Semantic Textual Similarity (STS) tasks further validated the approach, as the system outperformed the vast majority of participants in the 2013 *SEM STS competition. Ablation studies highlighted that contextual features are often more influential than purely lexical ones, and that the global refinement in Stage 2 is essential for maintaining high recall.
Conclusion
The two-stage logistic regression approach demonstrates that monolingual alignment benefits significantly from a hierarchical processing of features. By first assessing local similarity and then resolving global interdependencies, the model effectively handles the complexities of semantic context. Future work will focus on extending this framework to handle long-distance relationships and more complex many-to-many phrasal alignments.
Summary updated