Abstract
While Large Language Models (LLMs) excel at generating fluent text, they often struggle with logical consistency and rhetorical control in automated argumentation. We introduce ArgU, a framework designed to move neural argument generation toward logical and factual "controllability." By leveraging classical argumentation theory and a novel two-step generation process, ArgU produces arguments that follow specific reasoning patterns. Our approach is supported by a new corpus of nearly 70,000 annotated arguments, the largest of its kind for identifying complex reasoning patterns.
Introduction
The core challenge in automated argumentation is controllability. Most existing generators allow users to specify a topic or sentiment, but they lack the precision to dictate how an argument is constructed—for instance, arguing based on expert authority versus potential consequences. Without this control, models frequently produce illogical leaps or factual hallucinations. ArgU addresses this by using "argument templates" and control codes to dictate both the stance (pro/con) and the logical structure, known as argument schemes.
Methodology: The ArgU Framework
The ArgU system employs a multi-stage approach involving data curation and a novel dual-model architecture.
1. Argument Understanding
Before generation, the system must understand existing arguments. We developed a RoBERTa-based tagger, ArgSpan, to identify factual spans in text, and ArgSpanScheme, a classification head that identifies which of six Walton’s argument schemes (e.g., Source Authority, From Consequence, Rule or Principle) the text follows.
2. ArgU-Dual Generation
Unlike standard "one-shot" generators, ArgU-Dual utilizes a two-step process:
- Step 1: Template Generation. Given a set of facts and control codes (stance + scheme), the model generates an argument template—a logical skeleton with placeholders (e.g.,
<VAR_1> is known to <VAR_2>). - Step 2: Content Filling. The model fills this template with specific real-world concepts to produce a final, coherent argument.
Conditioning the model on special tokens like <pro>, <con>, and scheme-specific tags allows for the generation of multiple distinct arguments from the same input facts by simply swapping control codes.
The ArgU Corpus
To train and evaluate these models, we introduced the ArgU Corpus, a significant new resource for the NLP community. It contains 69,428 arguments spanning six controversial topics: abortion (AB), minimum wage (MW), nuclear energy (NE), gun control (GC), death penalty (DP), and school uniforms (SU).
The dataset was created using a "silver-standard" approach, where a small human-annotated set (1,153 examples) was used to train the ArgSpanScheme model, which then automatically annotated a much larger parallel corpus. This represents the largest publicly available corpus for identifying argument schemes to date.
Key Insights and Results
Our evaluation revealed several critical findings regarding the intersection of logic and neural generation:
- Logic as Scaffolding: Generating a template first acts as a logical constraint. By separating structure from content, the model significantly reduces illogical "hallucinations."
- Scheme Invariance: Argument schemes are largely topic-independent. An argument "From Consequence" follows the same logical flow across different subjects. Masking specific facts during training allows the model to learn these abstract reasoning patterns.
- Performance: ArgU-Dual outperformed single-step "Mono" versions in automatic metrics, achieving a BLEU score of 0.158 and a Rouge-L of 0.381. Human evaluations (on a 1-5 scale) rated the model highly for Fluency (4.86) and Logical Coherence (4.06).
- Fact Faithfulness: The model maintained high semantic similarity (~0.64) between input facts and output text, demonstrating successful knowledge incorporation.
Limitations and Future Directions
While ArgU represents a significant step forward, several challenges remain. The model occasionally struggles with linguistic nuances, such as correctly inverting logic when an input variable contains negative verbs (e.g., "stop"). Additionally, "template drifting" can occur in the second generation step, where the decoder modifies the template, potentially altering the intended logical scheme. Future work will focus on expanding the system's zero-shot capabilities to new topics and extending the generator to handle longer-form rhetorical synthesis beyond the current 50-token limit.
Summary updated