"You are no Jack Kennedy": On Media Selection of Highlights from Presidential Debates

Chenhao Tan, Hao Peng, Noah A. Smith

TL;DR

This paper investigates the "secret sauce" behind why certain sentences from presidential debates become media highlights while others are ignored. By analyzing three decades of debate transcripts and news coverage, the researchers found that while humans struggle to predict what will be quoted, machine learning models can identify linguistic and contextual signals—like "negativity bias" and shifts in conversational flow—that drive media selection.

What Problem Does This Paper Address?

Most voters don't watch presidential debates in their entirety; instead, they consume "highlights" curated by news outlets. These selected snippets—like the famous "You are no Jack Kennedy" retort—shape public perception and political reality.

However, the criteria for this selection process have remained largely qualitative or anecdotal. The authors seek to quantify the factors that make a sentence "quotable." They address three main questions:

  1. Can we predict which sentences will be quoted based on wording alone?
  2. How does human intuition about "newsworthiness" compare to data-driven models?
  3. Has the media's quoting behavior become more polarized or fragmented over time?

Methods

The researchers framed the problem as a binary classification task. They paired a "highlight" (a sentence quoted frequently in the week following a debate) with a "control" sentence (not quoted, but from the same speaker, of similar length, and occurring within three turns of the highlight).

Key technical approaches included:

  • Feature Engineering: They moved beyond simple bag-of-words to include:
  • Sentence-alone features: Length, sentiment (LIWC), personal pronouns, and "generality" (indefinite articles).
  • Conversation-flow features: Using Jaccard similarity to measure how a sentence relates to the speaker's previous turns and how it is "echoed" by other participants later in the debate.
  • Human Benchmarking: They recruited Mechanical Turk workers to perform the same classification task to see if "newsworthiness" is intuitive to the general public.
  • Bipartite Graph Analysis: To study media fragmentation, they built a graph connecting media outlets to the sentences they quoted and applied a min-cut algorithm to measure bipartisan coverage.

Datasets

The study utilizes a massive, newly compiled longitudinal dataset:

  • Debate Transcripts: Every U.S. presidential debate transcript from 1960 (general) and 2000 (primaries) through 2016.
  • News Coverage: Post-debate articles from approximately 700 unique newspapers indexed in LexisNexis (1980–2016).
  • The Result: A dataset of ~14,000 sentence pairs used for the classification experiments.

Key Insights and Ideas

  • The "Kink" in the Flow: One of the most striking findings is that highlights often represent a "changepoint" in a monologue. A quoted sentence is typically less similar to the speaker’s immediately preceding turn but more likely to be echoed in subsequent turns by others.
  • Negativity and Individualism: Media outlets show a strong "negativity bias," preferring sentences with negative emotions and negations. They also prefer "I/You/He/She" over the collective "We," suggesting a preference for individual conflict over unifying statements.
  • The Human-Machine Gap: Humans achieved only 60% accuracy in identifying highlights, often relying on "memory" of famous clips. In primary debates (where memory is weaker), machine classifiers significantly outperformed humans, suggesting that media selection follows linguistic patterns that aren't consciously salient to the public.
  • Front-loading: Quotes are not evenly distributed; the media is significantly more likely to quote from the first half of a debate than the second.

Key Results

  • Prediction Accuracy: The best-performing classifier (combining all features) achieved 66% accuracy, compared to the 60% human average.
  • Primary vs. General: Humans are much better at identifying highlights from general election debates than from primaries, likely due to higher past exposure.
  • Declining Bipartisanship: The min-cut analysis revealed a statistically significant downward trend in bipartisan coverage since 1992. Media outlets are increasingly quoting "their side" of the aisle or quoting the opposition in different (often more negative) contexts.
  • Local Clustering: While the media as a whole isn't necessarily becoming less similar, "local clusters" of media outlets are becoming tighter, indicating that specific groups of outlets are increasingly echoing the same set of highlights.

Limitations and Future Work

  • Text-Only Analysis: The study focuses purely on transcripts. It does not account for non-verbal cues like tone of voice, facial expressions, or "theatrical" moments (e.g., a candidate checking their watch), which are known to influence media selection.
  • Inter-media Influence: The model treats media outlets as independent actors, but in reality, outlets often quote what other outlets are already highlighting (the "echo chamber" effect).
  • External Factors: The paper acknowledges that events outside the debate (like a pre-existing scandal) heavily influence what the media looks for, which a text-only model cannot capture.

The authors suggest that future work could integrate audio-visual features and model the "diffusion" of quotes across different media tiers (e.g., from Twitter to cable news to newspapers).

Summary updated

Abstract

Political speeches and debates play an important role in shaping the images of politicians, and the public often relies on media outlets to select bits of political communication from a large pool of utterances. It is an important research question to understand what factors impact this selection process. To quantitatively explore the selection process, we build a three- decade dataset of presidential debate transcripts and post-debate coverage. We first examine the effect of wording and propose a binary classification framework that controls for both the speaker and the debate situation. We find that crowdworkers can only achieve an accuracy of 60% in this task, indicating that media choices are not entirely obvious. Our classifiers outperform crowdworkers on average, mainly in primary debates. We also compare important factors from crowdworkers» free-form explanations with those from data-driven methods and find interesting differences. Few crowdworkers mentioned that "context matters", whereas our data show that well-quoted sentences are more distinct from the previous utterance by the same speaker than less-quoted sentences. Finally, we examine the aggregate effect of media preferences towards different wordings to understand the extent of fragmentation among media outlets. By analyzing a bipartite graph built from quoting behavior in our data, we observe a decreasing trend in bipartisan coverage.
BibTeX
@inproceedings{tan2018you,
  author = {Chenhao Tan and Hao Peng and Noah A. Smith},
  title = {{"You are no Jack Kennedy": On Media Selection of Highlights from Presidential Debates}},
  booktitle = {The Web Conference},
  year = {2018},
  url = {https://openreview.net/forum?id=Hk4n1fb_Wr},
}

Loading related content...

Figure