"You are no Jack Kennedy": On Media Selection of Highlights from Presidential Debates

Chenhao Tan, Hao Peng, Noah A. Smith

Abstract

Political speeches and debates play an important role in shaping the images of politicians, and the public often relies on media outlets to select bits of political communication from a large pool of utterances. It is an important research question to understand what factors impact this selection process. To quantitatively explore the selection process, we build a three-decade dataset of presidential debate transcripts and post-debate coverage. We first examine the effect of wording and propose a binary classification framework that controls for both the speaker and the debate situation. We find that crowdworkers can only achieve an accuracy of 60% in this task, indicating that media choices are not entirely obvious. Our classifiers outperform crowdworkers on average, mainly in primary debates. We also compare important factors from crowdworkers' free-form explanations with those from data-driven methods and find interesting differences. Few crowdworkers mentioned that "context matters", whereas our data show that well-quoted sentences are more distinct from the previous utterance by the same speaker than less-quoted sentences. Finally, we examine the aggregate effect of media preferences towards different wordings to understand the extent of fragmentation among media outlets. By analyzing a bipartite graph built from quoting behavior in our data, we observe a decreasing trend in bipartisan coverage.

Introduction

Most voters do not watch presidential debates in their entirety; instead, they consume "highlights" curated by news outlets. These selected snippets—like the famous "You are no Jack Kennedy" retort—shape public perception and political reality. However, the criteria for this selection process have remained largely qualitative or anecdotal. This study seeks to quantify the factors that make a sentence "quotable," addressing whether highlights can be predicted by wording alone, how human intuition compares to data-driven models, and whether media quoting behavior has become more polarized over time.

Methods

The researchers framed the problem as a binary classification task, pairing a "highlight" (a sentence quoted frequently in the week following a debate) with a "control" sentence from the same speaker, of similar length, and occurring within three turns of the highlight.

Technical approaches included:

  • Feature Engineering: Moving beyond simple bag-of-words to include sentence-alone features (length, sentiment, pronouns) and conversation-flow features (using Jaccard similarity to measure how a sentence relates to the speaker's previous turns and subsequent "echoes" from others).
  • Human Benchmarking: Recruiting Mechanical Turk workers to perform the same classification task to evaluate the intuitiveness of "newsworthiness."
  • Bipartite Graph Analysis: Applying a min-cut algorithm to a graph connecting media outlets to the sentences they quoted to measure bipartisan coverage trends.

Key Insights and Ideas

The "Kink" in the Flow

A striking finding is that highlights often represent a "changepoint" in a monologue. A quoted sentence is typically less similar to the speaker’s immediately preceding turn but more likely to be echoed in subsequent turns by other participants. This suggests that media outlets prefer sentences that break the existing flow of conversation to introduce a new point or a sharp retort.

Figure 5: Jaccard similarity of highlighted vs. non-highlighted sentences to the speaker's own previous turns.

Front-loading and Negativity

Quotes are not evenly distributed throughout a debate. Media outlets are significantly more likely to quote from the first half of a debate than the second, a phenomenon termed "front-loading." Additionally, there is a strong "negativity bias," with a preference for sentences containing negative emotions and negations. Outlets also favor individualistic pronouns ("I/You/He/She") over collective ones ("We"), highlighting a preference for individual conflict over unifying statements.

Figure 3: Distribution of quotes across the progress of a debate, showing a clear front-loading effect.

The Human-Machine Gap

Humans achieved only 60% accuracy in identifying highlights, often relying on memory of famous clips. In primary debates, where memory is weaker, machine classifiers (achieving 66% accuracy) significantly outperformed humans. This suggests that media selection follows linguistic patterns that are not consciously salient to the general public.

Key Results

  • Prediction Accuracy: The best-performing classifier achieved 66% accuracy, outperforming the 60% human average.
  • Declining Bipartisanship: Analysis revealed a statistically significant downward trend in bipartisan coverage since 1992. Media outlets are increasingly quoting "their side" of the aisle or quoting the opposition in different, often more negative, contexts.
  • Local Clustering: While the media as a whole isn't necessarily becoming less similar, specific groups of outlets are increasingly echoing the same set of highlights, indicating tighter "local clusters."

Limitations and Future Work

The study focuses purely on transcripts and does not account for non-verbal cues like tone of voice or facial expressions. Furthermore, it treats media outlets as independent actors, though they often influence each other in an "echo chamber" effect. Future work could integrate audio-visual features and model the diffusion of quotes across different media tiers, such as from social media to cable news.

Summary updated

Abstract

Political speeches and debates play an important role in shaping the images of politicians, and the public often relies on media outlets to select bits of political communication from a large pool of utterances. It is an important research question to understand what factors impact this selection process. To quantitatively explore the selection process, we build a three- decade dataset of presidential debate transcripts and post-debate coverage. We first examine the effect of wording and propose a binary classification framework that controls for both the speaker and the debate situation. We find that crowdworkers can only achieve an accuracy of 60% in this task, indicating that media choices are not entirely obvious. Our classifiers outperform crowdworkers on average, mainly in primary debates. We also compare important factors from crowdworkers' free-form explanations with those from data-driven methods and find interesting differences. Few crowdworkers mentioned that "context matters", whereas our data show that well-quoted sentences are more distinct from the previous utterance by the same speaker than less-quoted sentences. Finally, we examine the aggregate effect of media preferences towards different wordings to understand the extent of fragmentation among media outlets. By analyzing a bipartite graph built from quoting behavior in our data, we observe a decreasing trend in bipartisan coverage.
BibTeX
@inproceedings{tan2018you,
  author = {Chenhao Tan and Hao Peng and Noah A. Smith},
  title = {{"You are no Jack Kennedy": On Media Selection of Highlights from Presidential Debates}},
  booktitle = {The Web Conference},
  year = {2018},
  url = {https://openreview.net/forum?id=Hk4n1fb_Wr},
}

Loading related content...

Figure