Abstract
Modern language models are often criticized for their "black-box" nature, where reasoning is hidden within millions of parameters. This paper introduces a novel framework that forces models to generate human-readable textual hypotheses before selecting an answer in multi-choice question-answering (MC-QA) tasks. By utilizing a differentiable generator-classifier architecture, the proposed model achieves performance parity with standard end-to-end models while providing a transparent window into its internal knowledge and failure modes.
Introduction
Large pre-trained language models (LMs) like BERT or ALBERT are incredibly proficient at answering questions, but they are notoriously opaque. In a standard "end-to-end" architecture, the reasoning happens entirely within hidden mathematical parameters. This creates two major issues: a lack of trust, as we cannot verify if the model is using statistical shortcuts, and difficulty in debugging, as it is nearly impossible to tell if a wrong answer stems from a lack of knowledge or a misunderstanding of the prompt.
Methods
The authors propose a two-part architecture designed to make the reasoning process explicit:
- The Generator: An LM (such as GPT-2 or XLNet) that takes the question and generates a "textual hypothesis"—a few natural language words representing the knowledge needed to answer.
- The Classifier: A second model that takes this hypothesis to select the final answer from the multiple-choice options.
To train the Generator and Classifier together, the researchers used the Gumbel-Softmax straight-through estimator. This technique allows the model to treat discrete word choices as continuous during training, enabling gradient-based learning. To ensure the generated text remained human-readable, they incorporated a KLD Loss to prevent straying from natural language patterns and a Repetition Penalty to stop the model from repeating keywords to "shout" at the classifier.
Experimental Results
The model was evaluated on three distinct tasks: Synthetic Hypernym Extraction, CommonsenseQA (CSQA), and QASC (science-based questions). The latter was used to test zero-shot transfer—how well the model generalizes to new domains without extra training.
Key results include:
- Performance Parity: On the CSQA dataset, the best explainable model achieved 70.9% accuracy, virtually identical to the 71.0% accuracy of the standard "black-box" baseline.
- Generalization: When tested on the QASC dataset without fine-tuning, the model reached 39.2%, actually outperforming the end-to-end baseline (38.9%).
- Human Approval: In blind tests, human raters gave the model's explanations a "reasonableness" score of 0.74 out of 1.0, indicating the generated text was largely logical.
Discussion and Insights
The generated hypotheses revealed why the model failed in certain instances. For example, for a question about a "hostess," the model generated "dinner eat serve," showing it confused the role with that of a waitress. This level of insight is impossible in standard models.
The researchers also noted a trade-off between utility and naturalness. Models trained strictly for accuracy often produce "list-like" keywords, while those forced to produce grammatically perfect sentences perform slightly worse on the actual QA task. Furthermore, to prevent the classifier from "cheating" using its own internal weights, the authors found that a simple similarity classifier was most effective at forcing the generator to produce semantically relevant words.
Conclusion
This framework demonstrates that explainability does not have to come at the cost of performance. While the current hypotheses often resemble "bags of words," future work will focus on encouraging more complex syntactic structures and reducing the classifier's reliance on hidden reasoning.
Summary updated