Abstract
Political speeches and debates play an important role in shaping the images of politicians, and the public often relies on media outlets to select bits of political communication from a large pool of utterances. It is an important research question to understand what factors impact this selection process. To quantitatively explore the selection process, we build a three-decade dataset of presidential debate transcripts and post-debate coverage. We first examine the effect of wording and propose a binary classification framework that controls for both the speaker and the debate situation. We find that crowdworkers can only achieve an accuracy of 60% in this task, indicating that media choices are not entirely obvious. Our classifiers outperform crowdworkers on average, mainly in primary debates. We also compare important factors from crowdworkers' free-form explanations with those from data-driven methods and find interesting differences. Few crowdworkers mentioned that "context matters", whereas our data show that well-quoted sentences are more distinct from the previous utterance by the same speaker than less-quoted sentences. Finally, we examine the aggregate effect of media preferences towards different wordings to understand the extent of fragmentation among media outlets. By analyzing a bipartite graph built from quoting behavior in our data, we observe a decreasing trend in bipartisan coverage.
Introduction
Most voters do not watch presidential debates in their entirety; instead, they consume "highlights" curated by news outlets. These selected snippets—like the famous "You are no Jack Kennedy" retort—shape public perception and political reality. However, the criteria for this selection process have remained largely qualitative or anecdotal. This study seeks to quantify the factors that make a sentence "quotable," addressing whether highlights can be predicted by wording alone, how human intuition compares to data-driven models, and whether media quoting behavior has become more polarized over time.
Methods
The researchers framed the problem as a binary classification task, pairing a "highlight" (a sentence quoted frequently in the week following a debate) with a "control" sentence from the same speaker, of similar length, and occurring within three turns of the highlight.
Technical approaches included:
- Feature Engineering: Moving beyond simple bag-of-words to include sentence-alone features (length, sentiment, pronouns) and conversation-flow features (using Jaccard similarity to measure how a sentence relates to the speaker's previous turns and subsequent "echoes" from others).
- Human Benchmarking: Recruiting Mechanical Turk workers to perform the same classification task to evaluate the intuitiveness of "newsworthiness."
- Bipartite Graph Analysis: Applying a min-cut algorithm to a graph connecting media outlets to the sentences they quoted to measure bipartisan coverage trends.
Key Insights and Ideas
The "Kink" in the Flow
A striking finding is that highlights often represent a "changepoint" in a monologue. A quoted sentence is typically less similar to the speaker’s immediately preceding turn but more likely to be echoed in subsequent turns by other participants. This suggests that media outlets prefer sentences that break the existing flow of conversation to introduce a new point or a sharp retort.
Front-loading and Negativity
Quotes are not evenly distributed throughout a debate. Media outlets are significantly more likely to quote from the first half of a debate than the second, a phenomenon termed "front-loading." Additionally, there is a strong "negativity bias," with a preference for sentences containing negative emotions and negations. Outlets also favor individualistic pronouns ("I/You/He/She") over collective ones ("We"), highlighting a preference for individual conflict over unifying statements.
The Human-Machine Gap
Humans achieved only 60% accuracy in identifying highlights, often relying on memory of famous clips. In primary debates, where memory is weaker, machine classifiers (achieving 66% accuracy) significantly outperformed humans. This suggests that media selection follows linguistic patterns that are not consciously salient to the general public.
Key Results
- Prediction Accuracy: The best-performing classifier achieved 66% accuracy, outperforming the 60% human average.
- Declining Bipartisanship: Analysis revealed a statistically significant downward trend in bipartisan coverage since 1992. Media outlets are increasingly quoting "their side" of the aisle or quoting the opposition in different, often more negative, contexts.
- Local Clustering: While the media as a whole isn't necessarily becoming less similar, specific groups of outlets are increasingly echoing the same set of highlights, indicating tighter "local clusters."
Limitations and Future Work
The study focuses purely on transcripts and does not account for non-verbal cues like tone of voice or facial expressions. Furthermore, it treats media outlets as independent actors, though they often influence each other in an "echo chamber" effect. Future work could integrate audio-visual features and model the diffusion of quotes across different media tiers, such as from social media to cable news.
Summary updated