Think Rationally about What You See: Continuous Rationale Extraction for Relation Extraction

Xuming Hu, Zhaochen Hong, Chenwei Zhang, Irwin King, Philip S. Yu

Abstract

Relation extraction (RE) is a fundamental task in information retrieval that aims to identify relationships between entities within a sentence. However, standard models often struggle with contextual noise—irrelevant words that mislead the reasoning process. We introduce $RE^2$, a novel framework that leverages continuous rationale extraction to filter out noisy segments while preserving semantic coherence. By applying an optimizable binary mask constrained by continuity and sparsity, $RE^2$ identifies the most informative text segments for relation classification. Our experiments across four major benchmarks demonstrate that $RE^2$ achieves state-of-the-art performance, particularly in low-resource scenarios, while providing human-interpretable explanations for its predictions.

Introduction: The Challenge of Contextual Noise

The primary goal of Relation Extraction (RE) is to determine the relationship between two entities based on their context. For instance, in the sentence "Ryan and Yaz knew each other from school," the relation between Ryan and Yaz is alumni.

A significant hurdle in this task is contextual noise. Sentences often contain "distractor" words that lead models astray. Consider a sentence like: "Ryan and Yaz... were similar to me and my friend." A standard model might focus on the word "friend" and incorrectly predict the relation as peer instead of alumni.

Comparison of Entity, Counterfactual, and Rational Thinking approaches in Relation Extraction.

Existing methods, such as "Entity Thinking" (focusing on entity types) or "Counterfactual Thinking" (removing model bias), do not explicitly discard this noisy content. Furthermore, previous attempts at rationale extraction often resulted in fragmented, unreadable snippets that destroyed the semantic flow necessary for modern language models like BERT to function effectively.

The $RE^2$ Framework

To address these issues, we propose the Continuous Rationale Extractor. This module applies an optimizable binary mask to every token in a sentence. To ensure the extracted rationales are useful and readable, we introduce two primary constraints via a factor graph:

  1. Continuity (CON): This factor encourages the model to select consecutive tokens. By favoring sequences over isolated words, the extracted rationale remains fluent and semantically coherent, which is vital for maintaining the performance of the downstream BERT-based classifier.
  2. Sparsity (SPA): This factor imposes a limit ($K$) on the total number of tokens selected. This forces the model to be "rational" and discard irrelevant segments, keeping only the most critical information.

Training via Latent Variables

Because "gold labels" for rationales do not exist, we treat rationale selection as a latent variable problem. We utilize Marginal Inference with Lagrange Multipliers and a sampling strategy called Perturb-and-MAP to make the selection process differentiable. This allows the rationale extractor and the relation classifier to be trained together end-to-end.

Experimental Evaluation

We evaluated $RE^2$ on four standard RE benchmarks: SemEval-2010 Task 8, TACRED, TACRED-Revisit, and Re-TACRED. To test robustness, we also conducted experiments in low-resource settings, using only 10%, 25%, and 50% of the training data.

Key Results

  • State-of-the-Art Performance: $RE^2$ outperformed previous SOTA models, including UNIREX and CORE, across all four datasets.
  • Regularization in Low-Resource Settings: In the 10% data setting, $RE^2$ showed a 1.1% F1 improvement compared to 0.9% in full-data settings. This suggests that forcing the model to find rationales acts as a powerful regularizer, preventing it from over-fitting to spurious correlations.
  • Human Validation: In a human study, annotators achieved higher accuracy using only the extracted rationales (95.5% on SemEval) than they did using the original, full sentences (94.3%). This confirms that the model successfully isolates the most relevant information while removing confusing noise.
  • Interpretability: Unlike "black box" models, $RE^2$ provides a clear visual mask of the evidence it used to make a decision, making the logic transparent to users.

Conclusion and Future Work

$RE^2$ demonstrates that "thinking rationally" by filtering noise through continuous and sparse rationale extraction significantly improves both the performance and interpretability of relation extraction models. While the sparsity parameter ($K$) currently requires manual tuning, future work will explore automated methods for determining optimal sparsity based on sentence complexity. We also plan to extend this framework to Knowledge Graph construction and other information retrieval tasks such as document ranking and question answering.

Summary updated

Abstract

Relation extraction (RE) aims to extract potential relations according to the context of two entities, thus, deriving rational contexts from sentences plays an important role. Previous works either focus on how to leverage the entity information (e.g., entity types, entity verbalization) to inference relations, but ignore context-focused content, or use counterfactual thinking to remove the model's bias of potential relations in entities, but the relation reasoning process will still be hindered by irrelevant content. Therefore, how to preserve relevant content and remove noisy segments from sentences is a crucial task. In addition, retained content needs to be fluent enough to maintain semantic coherence and interpretability. In this work, we propose a novel rationale extraction framework named RE2, which leverages two continuity and sparsity factors to obtain relevant and coherent rationales from sentences. To solve the problem that the gold rationales are not labeled, RE2 applies an optimizable binary mask to each token in the sentence, and adjust the rationales that need to be selected according to the relation label. Experiments on four datasets show that RE2 surpasses baselines.
BibTeX
@inproceedings{hu2023think,
  author = {Xuming Hu and Zhaochen Hong and Chenwei Zhang and Irwin King and Philip S. Yu},
  title = {{Think Rationally about What You See: Continuous Rationale Extraction for Relation Extraction}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2023},
  doi = {10.1145/3539618.3592072},
}
Figure