Abstract
Stance detection (SD) often struggles when input text lacks explicit links to its target or relies on implicit commonsense knowledge. We present INJECT, a dual-encoder framework that retrieves external knowledge from large language models (LLMs) or knowledge bases and integrates it via cross-attention. Our approach demonstrates significant performance improvements across 16 diverse datasets and exhibits superior robustness to noisy or irrelevant context compared to standard text concatenation methods.
Introduction
Stance detection involves determining an author’s position—Pro, Anti, or Neutral—toward a specific target. Unlike sentiment analysis, SD is challenging because targets are frequently unmentioned, or the reasoning requires external context. For instance, identifying a "Pro" stance on "School Uniforms" from the statement "Uniforms create school spirit" requires knowing that "school spirit" is a positive attribute. Existing models often fail in cross-target scenarios or rely on spurious correlations rather than logical argumentation.
The INJECT Framework
To address these challenges, we propose INJECT, a dual-encoder architecture that weaves external context into the model's decision-making process.
The framework consists of two parallel encoders: one for the input text and target, and another for retrieved contextual information. Instead of simple concatenation, the "Inject Block" utilizes cross-attention at specific transformer layers. This allows the input encoder to attend to context representations, enabling the model to re-weigh token importance based on external knowledge. By treating all external knowledge—whether from structured knowledge bases like ConceptNet or LLM-generated definitions—as natural language, the system remains source-agnostic.
Experimental Setup
The method was evaluated on a comprehensive benchmark of 16 stance detection datasets spanning various domains, including debate fora (arc, iac1, perspectrum), news (emergent, fnc1, snopes), and social media (mtsd, rumor, wtwt). Retrieval strategies included structured knowledge triplets converted to sentences and LLM prompting (using T0pp) to generate explanations for targets and key noun phrases.
Results and Discussion
INJECT outperformed the strong BERT+Target baseline in 13 out of 16 datasets. Notably, on the rumor dataset, it achieved an 11.7 percentage point improvement in macro-F1. While the framework is model-agnostic, the highest overall performance was observed when using ELECTRA as the base model.
Robustness and Regularization
A key finding is INJECT's robustness to noise. While simple concatenation often degrades performance if context is irrelevant, INJECT's separate encoding and cross-attention mechanism act as a filter. Even with random context, INJECT remained stable. Furthermore, analysis suggests that INJECT serves as a regularizer, encouraging the model to ignore spurious target-specific keywords and focus on label-indicative tokens, thereby improving reasoning capabilities.
Conclusion
The INJECT framework provides a robust and flexible method for integrating external knowledge into stance detection. While it introduces computational overhead due to the dual-encoder setup, its ability to leverage heterogeneous knowledge sources and maintain performance in the presence of noise makes it a significant advancement for cross-target stance detection. Future work will explore parameter-efficient architectures and automated prompt optimization to further enhance efficiency and context quality.
Summary updated