Abstract
Po-Nien Kung, Tse-Hsuan Yang, Yi-Cheng Chen, Sheng-Siang Yin, Yun-Nung Chen. Findings of the Association for Computational Linguistics: EMNLP 2020.
2020.
Introduction
Deep learning models are often criticized as "black boxes." To establish trust, particularly in high-stakes domains like medicine or finance, models must provide rationales: specific snippets of input text that justify a prediction. However, obtaining rationale annotations is labor-intensive and expensive compared to simple labels. While supervised methods exist, they rely on these rare annotations. This paper addresses the challenge by proposing a zero-shot rationalization framework that leverages multi-task transfer learning from Question Answering (QA).
Methodology: Rationalization as Question Answering
The core innovation is treating the identification of a rationale as a QA task. The authors hypothesize that a rationale is essentially the "answer" to the question "Why was this label chosen?"
Shared Architecture and Multi-Task Training
The framework utilizes a shared ALBERT encoder with multiple task-specific heads: one for extractive QA (trained on SQuAD 2.0) and others for domain-specific classification (e.g., Beer or Movie ratings). To align these tasks, classification is reformulated into a QA format. For instance, a beer review is paired with a question like "What is this beer appearance score?"
The model undergoes Multi-Task Learning (MTL), cycling through QA and classification tasks. This forces the encoder to develop a universal understanding of how specific text spans relate to queries. During inference, the model uses its QA head to extract rationales from the target domain, despite never having seen a human-annotated rationale for that domain during training.
Experimental Results
The model was evaluated on the BeerReview and MovieReview datasets, using SQuAD 2.0 as the source for teaching reading comprehension mechanics.
Performance on BeerReview
The proposed multi-task approach (S+B+M, combining SQuAD, Beer, and Movie data) achieved significant improvements in precision. As shown in the results table, it reached 93.41% precision at a 10% rationale length, outperforming previous state-of-the-art methods.
Generalization and Stability
In the MovieReview task, the zero-shot model achieved a 9.3 IOU F1 score, notably surpassing supervised baselines like "Bert-To-Bert" (7.5) which had access to rationale labels. Furthermore, the MTL approach demonstrated remarkable stability across different review aspects (e.g., aroma vs. palate), suggesting a robust, generalized understanding of evidence extraction.
Conclusion
This work demonstrates that rationalization can be effectively framed as a transfer learning problem from QA. By training on general reading comprehension tasks alongside classification, models can learn to extract high-quality, salient rationales in a zero-shot manner. While currently limited to extractive rationalization, this approach offers a scalable path toward more interpretable AI without the need for exhaustive rationale annotations.
Summary updated