TL;DR
This paper introduces JOA-ICL, a novel framework for article-level news stance detection that leverages "journalism-guided" segments (like headlines and quotes) to improve accuracy, especially for long-form Korean news articles. They also release K-NEWS-STANCE, the first Korean dataset for this task, addressing a critical gap in multilingual stance detection.
What Problem Does This Paper Address?
In today's digital age, personalized news recommendation systems are ubiquitous. However, they often create "filter bubbles" by reinforcing existing viewpoints, leading to political polarization. Stance detection – identifying a text's position (supportive, neutral, or oppositional) on a given issue – is crucial for mitigating this by enabling more diverse recommendations and uncovering media bias. Existing research, however, is largely limited to short texts (like tweets) and high-resource languages (like English), leaving a significant gap for longer, nuanced news articles in other languages, particularly Korean. The complexity of journalistic writing, where stances are often subtly embedded rather than explicitly stated, further complicates this task.
Methods
The core innovation is JOA-ICL (Journalism-Guided Agentic In-Context Learning). It's an "agentic" approach where a primary Large Language Model (LLM) for article-level stance detection delegates the task of predicting stances for specific, journalism-guided segments (headlines, leads, conclusions, and quotations) to a separate LLM "agent." These segment-level predictions are then integrated into the primary LLM's prompt, providing contextual cues to improve the final article-level stance prediction. This hierarchical approach aims to capture dispersed stance signals across different parts of a news article.
Datasets
The paper introduces K-NEWS-STANCE, the first Korean dataset for article-level stance detection. It comprises 2,000 news articles covering 47 societal issues, with both article-level and 21,650 segment-level stance annotations. This dataset is crucial for addressing the lack of resources in non-major languages. They also conducted experiments on CheeSE, a German-language dataset for article-level stance detection, to demonstrate the generalizability of their method.
Key Insights and Ideas
- Segment-level agency is key: The paper strongly suggests that breaking down long articles into journalistically significant segments and analyzing their stances individually significantly boosts the accuracy of overall article stance detection.
- Journalistic structure matters: Not all segments are created equal. Headlines, leads, and conclusions seem to carry more weight in signaling an article's overall stance compared to individual quotations, which can sometimes be more nuanced or even contradictory.
- LLMs can be specialized agents: The "agentic" approach, where one LLM delegates tasks to another, shows promise for complex NLP problems, allowing for specialized processing of different text components.
- Korean news is complex: The K-NEWS-STANCE dataset highlights the challenges of stance detection in Korean journalism, where stances can be subtle and require a deep understanding of context.
Key Results
JOA-ICL consistently outperformed existing stance detection methods, including chain-of-thought (CoT) prompting baselines, by up to +0.071 in accuracy and +0.1 in F1. When using a fine-tuned RoBERTa model as the segment-level agent, JOA-ICL achieved the highest accuracy on the K-NEWS-STANCE test set. The method also demonstrated strong generalizability, outperforming baselines on the German CheeSE dataset. Ablation studies revealed that removing the "lead" segment had the most significant negative impact on performance, underscoring its importance.
Limitations and Future Work
The authors acknowledge that their approach incurs additional inference costs due to the invocation of an LM agent at the segment level. They suggest that techniques like post-training quantization could help reduce this overhead. Future work could involve extending the agentic in-context learning method to multi-agent systems, perhaps using a debate-style framework. Further validation across more languages and media ecosystems is also recommended. The paper also notes that while JOA-ICL is effective, identifying "supportive" stances remains a challenge, warranting further investigation.
Summary updated