Abstract
In news reporting, information is rarely presented as raw fact; instead, it is often filtered through multiple layers of attribution, creating a "telephone game" effect where statements are nested within various perspectives. This paper presents a supervised machine learning approach to automatically reconstruct the hierarchical structure of these perspective and speech expressions (pse's). By identifying the "immediate parent" for each expression, our system achieves a link accuracy of 78.3%. We demonstrate that while syntactic dependency provides a strong baseline, a model incorporating lexical, positional, and genre-specific features is necessary to navigate the complexities of journalistic prose and accurately map the chain of attribution.
1. Introduction
For Natural Language Processing (NLP) systems, the "indirection" inherent in news writing is a significant challenge. A typical sentence might involve a writer reporting on a spokesperson who is summarizing a report about a community's feelings. If a system extracts "Bob is unhappy" as a standalone fact without recognizing it is filtered through "Alice’s claim" and the "Writer’s report," it misses the potential for bias, garbling, or misinterpretation inherent in the chain.
This research tackles the challenge of identifying the hierarchical structure of pse's—essentially mapping the tree of "who is filtering whose message." A key insight of this work is that every sentence has an implicit "writer's speech event" at the top; even if no one is explicitly quoted, the writer remains the primary filter.
2. Methodology
We propose a supervised machine learning approach that moves beyond simple rule-based heuristics. The system operates in two main stages:
2.1 Binary Classification
The model evaluates every pair of pse’s in a sentence to decide if one is the "immediate parent" of the other. For example, in the phrase "Alice's claim that Bob was unhappy," the model attempts to predict that claim is the parent of unhappy.
2.2 Global Structure Building
The system then combines these pairwise decisions to build a global tree for the entire sentence, choosing the most confident parent for each expression to ensure a coherent hierarchy.
2.3 Feature Engineering
The classifier (decTree) utilizes several categories of features to make its determinations:
- Parse-based features: Using dependency trees to see if one expression syntactically "dominates" another.
- Positional features: The relative distance and linear order of expressions in the sentence.
- Lexical features: Identifying specific high-frequency "filtering" words like "said" or "according to."
- Genre-specific patterns: Journalistic writing often uses unique structures, such as trailing attributions ("..., Bob argued"), that standard parsers struggle with. We added specific features to capture these journalistic quirks.
3. Experimental Setup
The study utilizes the MPQA (formerly NRRC) corpus, consisting of 535 newswire documents manually annotated for opinions and sources. Evaluation focused on sentences containing at least two non-writer pse’s, representing the truly complex "telephone game" scenarios. We employed 10-fold cross-validation across 469 test documents.
4. Results and Discussion
The learning-based approach significantly outperformed both a simple baseline and a syntax-only heuristic, as shown in the table below:
| Metric | Simple Baseline (All to Writer) | Parse-based Heuristic | ML Approach (decTree) |
|---|---|---|---|
| Lin’s Score (Link Accuracy) | 65.57% | 71.64% | 78.30% |
| Perfect Sentence Accuracy | 36.02% | 45.37% | 54.52% |
4.1 Key Insights
- Syntax is Not Everything: While the hierarchy often follows grammatical structure, a purely parse-based heuristic only reaches 71.6% accuracy. The ML model's ability to weigh syntax against lexical and positional cues provides the necessary performance boost.
- The "Said" Bias: The model learned that certain words like "said" are extremely likely to be parents, which occasionally led to errors where nested opinions were incorrectly attached directly to the main speaker rather than an intermediate source.
- Complexity Scaling: Performance remains a challenge as complexity grows; accuracy dipped to 68.68% for sentences containing more than five pse’s.
5. Conclusion
Automatically mapping the hierarchy of perspectives is a critical step toward more nuanced sentiment analysis and information extraction. While our model shows strong performance in reconstructing these chains, future work should address error propagation from underlying syntactic parsers and integrate source identity resolution to fully map the "who" behind the "what."
Summary updated