Abstract
As Large Language Models (LLMs) become ubiquitous in content creation, understanding their internal conceptualization of human traits like persuasiveness is critical. This study investigates how GPT-4o modifies French public speaking transcripts to enhance or diminish their persuasive impact. By analyzing 45 linguistic features across four dimensions—Discourse, Lexical, Syntactic, and Psycholinguistic—the researchers found that GPT-4o predominantly employs a "surface-level" strategy. This approach prioritizes emotional vocabulary and rhetorical syntax over deep logical restructuring, aligning with the "peripheral route" of persuasion.
Introduction
While LLMs demonstrate remarkable fluency, their grasp of complex human communication nuances remains a subject of intense research. Most studies on persuasiveness in public speaking focus on non-verbal cues, leaving the textual dimension under-explored, particularly in non-English contexts. This paper addresses this gap by examining how GPT-4o "upgrades" or "downgrades" the persuasiveness of transcripts from the 3MT_French dataset (PhD in 180 Seconds competition). The goal is to decode the model's stylistic "tricks" and compare them to established rhetorical theories.
Methodology
The researchers employed a controlled generation protocol using GPT-4o to create two variants of 90 human speech transcripts: one optimized for high persuasiveness and one for low persuasiveness. To quantify the differences, a comprehensive linguistic framework was developed, incorporating standard metrics like lexical diversity and LIWC (Linguistic Inquiry and Word Count) categories, alongside novel hand-crafted features for rhetorical devices.
These features, detailed in Figure 1, allowed for a granular analysis of how the model manipulates discourse transitions, syntactic structures, and emotional resonance to achieve its goal.
Key Insights: The AI Persuasion Strategy
The analysis revealed several systematic shifts in how GPT-4o conceptualizes persuasive text:
1. The "Peripheral Route" to Persuasion
Consistent with the Elaboration Likelihood Model, GPT-4o’s strategy for "upgrading" speeches relies on surface-level appeal. Instead of strengthening the core evidence, the model significantly boosts the use of affective (emotional) lexicon. Words associated with positive emotions and intensity are increased to sway the audience through sentiment rather than just logic.
2. Syntactic Engagement
A major finding was the systematic shift in sentence types. To increase persuasion, GPT-4o frequently replaces declarative statements with interrogative (rhetorical questions), exclamatory, and imperative structures. This shift aims to transform a passive monologue into an engaging dialogue with the audience. Conversely, "downgraded" speeches were characterized by more neutral, declarative, and conditional phrasing, which projects less authority.
3. The AI "Cleaning" Effect
Interestingly, the model consistently reduced "content redundancy" in both upgraded and downgraded versions. While this makes the text "cleaner" and more lexically diverse, it suppresses rhetorical devices like anaphora (repetition at the start of sentences). In human public speaking, such repetitions are often vital for emphasis and memorability; GPT-4o’s tendency to avoid them suggests a bias toward written-style "efficiency" over oral-style impact.
Results and Statistical Significance
The researchers used the Mann-Whitney U test to identify which linguistic shifts were statistically significant. The results, summarized in Figure 2, highlight the divergent paths the model takes when prompted to modify persuasiveness.
As shown in Figure 2, upgraded speeches show a marked increase in "insight" and "causation" markers alongside emotional intensity, while downgraded speeches exhibit higher "tentative" language and a decrease in certainty.
Conclusion and Future Work
The study concludes that GPT-4o equates persuasiveness with a combination of emotional intensity, rhetorical syntax, and sophisticated (often less readable) vocabulary. While effective at mimicking the "surface" of a persuasive speech, the model’s tendency to "clean" text by removing repetitions highlights a potential disconnect between AI-generated text and the requirements of effective oral delivery. Future research should expand this analysis to other languages and model architectures to determine if these stylistic biases are universal across the current generation of LLMs.
Summary updated