Playing the Telephone Game: Determining the Hierarchical Structure of Perspective and Speech Expressions

Eric Breck, Claire Cardie

Abstract

In news reporting, information is rarely presented as raw fact; instead, it is often filtered through multiple layers of attribution, creating a "telephone game" effect where statements are nested within various perspectives. This paper presents a supervised machine learning approach to automatically reconstruct the hierarchical structure of these perspective and speech expressions (pse's). By identifying the "immediate parent" for each expression, our system achieves a link accuracy of 78.3%. We demonstrate that while syntactic dependency provides a strong baseline, a model incorporating lexical, positional, and genre-specific features is necessary to navigate the complexities of journalistic prose and accurately map the chain of attribution.

1. Introduction

For Natural Language Processing (NLP) systems, the "indirection" inherent in news writing is a significant challenge. A typical sentence might involve a writer reporting on a spokesperson who is summarizing a report about a community's feelings. If a system extracts "Bob is unhappy" as a standalone fact without recognizing it is filtered through "Alice’s claim" and the "Writer’s report," it misses the potential for bias, garbling, or misinterpretation inherent in the chain.

The hierarchical structure of perspective and speech expressions, illustrating the writer's implicit speech event as the root of all attributions.

This research tackles the challenge of identifying the hierarchical structure of pse's—essentially mapping the tree of "who is filtering whose message." A key insight of this work is that every sentence has an implicit "writer's speech event" at the top; even if no one is explicitly quoted, the writer remains the primary filter.

2. Methodology

We propose a supervised machine learning approach that moves beyond simple rule-based heuristics. The system operates in two main stages:

2.1 Binary Classification

The model evaluates every pair of pse’s in a sentence to decide if one is the "immediate parent" of the other. For example, in the phrase "Alice's claim that Bob was unhappy," the model attempts to predict that claim is the parent of unhappy.

2.2 Global Structure Building

The system then combines these pairwise decisions to build a global tree for the entire sentence, choosing the most confident parent for each expression to ensure a coherent hierarchy.

2.3 Feature Engineering

The classifier (decTree) utilizes several categories of features to make its determinations:

  • Parse-based features: Using dependency trees to see if one expression syntactically "dominates" another.
  • Positional features: The relative distance and linear order of expressions in the sentence.
  • Lexical features: Identifying specific high-frequency "filtering" words like "said" or "according to."
  • Genre-specific patterns: Journalistic writing often uses unique structures, such as trailing attributions ("..., Bob argued"), that standard parsers struggle with. We added specific features to capture these journalistic quirks.
A dependency parse tree for a sentence with nested perspectives, showing the syntactic relationships that the model must map to a conceptual hierarchy.

3. Experimental Setup

The study utilizes the MPQA (formerly NRRC) corpus, consisting of 535 newswire documents manually annotated for opinions and sources. Evaluation focused on sentences containing at least two non-writer pse’s, representing the truly complex "telephone game" scenarios. We employed 10-fold cross-validation across 469 test documents.

4. Results and Discussion

The learning-based approach significantly outperformed both a simple baseline and a syntax-only heuristic, as shown in the table below:

Metric Simple Baseline (All to Writer) Parse-based Heuristic ML Approach (decTree)
Lin’s Score (Link Accuracy) 65.57% 71.64% 78.30%
Perfect Sentence Accuracy 36.02% 45.37% 54.52%

4.1 Key Insights

  • Syntax is Not Everything: While the hierarchy often follows grammatical structure, a purely parse-based heuristic only reaches 71.6% accuracy. The ML model's ability to weigh syntax against lexical and positional cues provides the necessary performance boost.
  • The "Said" Bias: The model learned that certain words like "said" are extremely likely to be parents, which occasionally led to errors where nested opinions were incorrectly attached directly to the main speaker rather than an intermediate source.
  • Complexity Scaling: Performance remains a challenge as complexity grows; accuracy dipped to 68.68% for sentences containing more than five pse’s.

5. Conclusion

Automatically mapping the hierarchy of perspectives is a critical step toward more nuanced sentiment analysis and information extraction. While our model shows strong performance in reconstructing these chains, future work should address error propagation from underlying syntactic parsers and integrate source identity resolution to fully map the "who" behind the "what."

Summary updated

Abstract

News articles report on facts, events, and opinions with the intent of conveying the truth. However, the facts, events, and opinions appearing in the text are often known only secondor third-hand, and as any child who has played "telephone" knows, this relaying of facts often garbles the original message. Properly understanding the information filtering structures that govern the interpretation of these facts, then, is critical to appropriately analyzing them. In this work, we present a learning approach that correctly determines the hierarchical structure of information filtering expressions 78.30% of the time.
BibTeX
@inproceedings{breck2004playing,
  author = {Eric Breck and Claire Cardie},
  title = {{Playing the Telephone Game: Determining the Hierarchical Structure of Perspective and Speech Expressions}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2004},
}
Figure