Abstract
This paper considers the problem of automatically inducing role-semantic annotations in the FrameNet paradigm for new languages. We introduce a general framework for semantic projection which exploits parallel texts, is relatively inexpensive and can potentially reduce the amount of effort involved in creating semantic resources. We propose projection models that exploit lexical and syntactic information. Experimental results on an English-German parallel corpus demonstrate the advantages of this approach.
Introduction: The Resource Gap in Deep Semantics
Understanding "who did what to whom" is a cornerstone of advanced NLP tasks like question answering and machine translation. The FrameNet project provides a gold-standard framework for this, where "frames" represent situations (e.g., DEPARTING) and "frame elements" represent roles (e.g., THEME, SOURCE).
The primary challenge is that creating a FrameNet is an immense manual undertaking. While English has a robust FrameNet, most other languages have very little. This research addresses the "resource gap" by investigating whether semantic roles can be automatically projected from English to a target language (like German) using parallel corpora.
Methodology: Projection Models
The authors propose two main families of projection models to bridge the language gap.
Word-based Models
These rely on automatic word alignments (using GIZA++). If an English word is labeled as a frame-evoking element and aligns with a German word, the label is projected. To handle gaps in alignment, a "convex complementing" heuristic is used, which assumes semantic roles usually cover contiguous stretches of text and fills in the "holes" between aligned tokens.
Constituent-based Models
Recognizing that semantic roles usually correspond to syntactic units (like Noun Phrases), these models use parsers to identify constituents in both languages. They then use a similarity function (based on word overlap) to find the best match between an English constituent and a German one.
Several strategies were evaluated:
- Forward Alignment: Maps each source role to exactly one target constituent (conservative and precise).
- Backward Alignment: Iterates over target constituents to find their best source match (more flexible but noisier).
- Zero-skipping: A heuristic that skips projection if the alignment is too weak, prioritizing precision over recall.
Experimental Setup
The study uses the Europarl corpus, containing proceedings from the European Parliament. To evaluate the models, the authors manually annotated 1,140 English-German sentence pairs to create a gold standard benchmark, utilizing the English FrameNet and the German SALSA project for guidance.
Key Results and Insights
One of the most significant findings is that semantic roles are remarkably stable across translations. Human annotators achieved a 0.91 F-score in role agreement across languages, confirming that projection is a theoretically sound strategy.
Quantitatively, the models performed as follows:
- Baseline (Word-only): Achieved an F-score of 0.41 (0.46 with the convex heuristic).
- Best Constituent Model: The forward-alignment model using content-word overlap and zero-skipping reached an F-score of 0.65 (with a precision of 0.70).
While 0.65 is a strong result for an automated process, it still falls short of the human "upper bound" of 0.84 (the agreement between two different human annotators on the same task).
Limitations and Future Work
The authors identify several "semantic speedbumps":
- Alignment Gaps: Roles in English may not exist in the German translation due to stylistic choices.
- Structural Divergence: A concept expressed as a verb in English might be an adverb in German, breaking constituent mapping.
- Parser Errors: Constituent models are dependent on the accuracy of underlying syntactic parsers.
Future work suggests incorporating "empty categories" to handle implied words and using these projected labels to train standalone semantic parsers for the target language.
Summary updated