Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control

Mo Yu, Shiyu Chang, Yang Zhang, Tommi S. Jaakkola

Abstract

Selective rationalization has become a common mechanism to ensure that predictive models reveal how they use any available features. The selection identifies a subset of input features relevant for prediction, viewed as a cooperative game between a rationale generator and a predictor. However, this setting often suffers from a "blind" generator and a lack of control over information left in the rationale complement. We introduce an introspective model that incorporates predicted outcomes into the selection process and explicitly controls the rationale complement via an adversary. Our results show that these mechanisms maintain high predictive accuracy while leading to more comprehensive and human-aligned rationales.

Introduction: The Challenges of Cooperative Rationalization

In interpretable AI, selective rationalization aims to force models to "show their work" by highlighting key input phrases (the rationale) used for prediction. Despite its popularity, existing cooperative frameworks face two critical issues:

  1. The Blind Generator: Generators typically select rationales without knowing the final prediction, often failing to justify the actual model outcome.
  2. Degeneration (Cheating): Models often learn "shortcuts" or "secret codes"—such as using specific punctuation to signal a label—rather than extracting semantic meaning. This results in high accuracy but uninformative rationales.

Methodology: Introspection and Complement Control

To address these flaws, we propose a revised architecture involving an introspective generator and a three-player adversarial game.

The Introspective Generator

Unlike standard models where the generator only sees the input $X$, our introspective generator first runs a standard classifier to predict a "proxy label" $\tilde{y}(X)$. The generator then uses both $X$ and $\tilde{y}$ to select the rationale $R$. This ensures the selected text is aligned with the likely prediction.

The Three-Player Minimax Game

To prevent "secret codes," we introduce a Complement Predictor ($Pred^c$). The framework becomes a game between three players:

  • Generator: Aims to select a rationale $R$ that helps the primary Predictor while simultaneously minimizing the performance of the Complement Predictor.
  • Predictor: Aims to predict the label $Y$ using only $R$.
  • Complement Predictor: Aims to predict $Y$ using only the excluded text $R^c$.
Evolution of the rationalization framework: (a) standard cooperative model, (b) model with complement control, and (c) the proposed full model with an introspective generator and complement control.

By forcing the Complement Predictor to fail, the Generator is incentivized to move all predictive information into the rationale $R$, leaving only noise in the complement $R^c$.

Experimental Evaluation

We evaluated our framework on several benchmarks, including Multi-aspect and Single-aspect Beer Reviews, and the SemEval 2010 Task 8 for relation classification.

Key Results

  • Predictive Accuracy: On multi-aspect beer reviews, our full model (Introspective + Minimax) achieved 86.16% accuracy with only 10% text highlighted, outperforming the 82.05% of the baseline cooperative model.
  • Comprehensiveness and Human Alignment: Our method produces rationales that align more closely with human annotations. As shown in qualitative examples, the introspective model captures more relevant entities and context compared to previous methods.
Qualitative comparison of rationales generated by the baseline (Lei et al. 2016) versus the proposed Intros+minimax framework on relation classification tasks.

Conclusion

By rethinking the cooperative game as a three-player adversarial process with introspection, we can generate rationales that are both more accurate and more honest. While training these models requires careful stabilization due to the complexity of reinforcement learning, the resulting "comprehensiveness" constraint ensures that the AI's explanations are not just shortcuts, but meaningful representations of its decision-making process. Future work will explore applying this to more complex domains like medical diagnosis.

Summary updated

Abstract

Selective rationalization has become a common mechanism to ensure that predictive models reveal how they use any available features. The selection may be soft or hard, and identifies a subset of input features relevant for prediction. The setup can be viewed as a co-operate game between the selector (aka rationale generator) and the predictor making use of only the selected features. The co-operative setting may, however, be compromised for two reasons. First, the generator typically has no direct access to the outcome it aims to justify, resulting in poor performance. Second, there's typically no control exerted on the information left outside the selection. We revise the overall co-operative framework to address these challenges. We introduce an introspective model which explicitly predicts and incorporates the outcome into the selection process. Moreover, we explicitly control the rationale complement via an adversary so as not to leave any useful information out of the selection. We show that the two complementary mechanisms maintain both high predictive accuracy and lead to comprehensive rationales.
BibTeX
@inproceedings{yu2019rethinking,
  author = {Mo Yu and Shiyu Chang and Yang Zhang and Tommi S. Jaakkola},
  title = {{Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=Xh_K_HxT9J1},
}
Figure