Abstract
Many question answering (QA) tasks only provide weak supervision for how the answer should be computed. For example, TriviaQA answers are entities that can be mentioned multiple times in supporting documents, while DROP answers can be computed by deriving many different equations from numbers in the reference text. In this paper, we show it is possible to convert such tasks into discrete latent variable learning problems with a precomputed, task-specific set of possible "solutions" (e.g. different mentions or equations) that contains one correct option. We then develop a hard EM learning scheme that computes gradients relative to the most likely solution at each update. Despite its simplicity, we show that this approach significantly outperforms previous methods on six QA tasks, including absolute gains of 2–10%, and achieves the state-of-the-art on five of them. Using hard updates instead of maximizing marginal likelihood is key to these results as it encourages the model to find the one correct answer, which we show through detailed qualitative analysis.
The Challenge of Weak Supervision
In many modern QA tasks, we have the question and the final answer, but the path the model should take to reach that answer is unknown. This is referred to as "weak supervision."
Consider two common scenarios:
- Multi-mention Reading Comprehension (e.g., TriviaQA): An answer like "Robert Schumann" might appear multiple times in a document. However, only one specific mention typically provides the actual evidence needed to answer the question.
- Discrete Reasoning (e.g., DROP): To arrive at the answer "4," a model might calculate
40 - 36(the correct derivation based on the text) or41 - 37(a "spurious" solution that happens to yield the same result using unrelated numbers).
Standard approaches often use Maximum Marginal Likelihood (MML), which maximizes the sum of probabilities of all paths leading to the correct answer. The authors argue that MML is fundamentally flawed in this context because it encourages the model to spread its probability mass across many spurious paths rather than identifying the single true logical derivation.
Methodology: Discrete Hard EM
The authors propose a three-step Discrete Hard EM framework to address these issues:
- Precomputation of Solution Sets ($Z$): For every training example, a finite set of all possible "solutions" (spans, equations, or SQL queries) that result in the ground-truth answer is precomputed.
- Hard EM Update: During training, the model identifies the single solution $\tilde{z}$ in the set $Z$ that has the highest probability according to its current parameters.
- Optimization: The model performs a gradient update to maximize the likelihood of only that top-ranked solution. This "winner-take-all" strategy forces the model to commit to a single path.
To prevent the model from locking onto a spurious solution too early, an Annealing schedule is used. Early in training, the model behaves like MML to explore the space; as training progresses, it shifts toward Hard EM to refine its selection.
Key Results and Robustness
The approach was evaluated on six datasets, including TriviaQA, DROP, and WikiSQL. The Discrete Hard EM approach consistently outperformed both simple heuristics and MML, achieving state-of-the-art results on five of the six benchmarks.
A critical finding is the model's robustness to noise. As the number of possible solutions ($|Z|$) increases, the likelihood of including spurious paths grows. While MML's performance degrades significantly as $|Z|$ increases, the Hard EM approach maintains high accuracy.
Key Insights:
- Alignment of Training and Inference: MML trains on a sum of probabilities, but inference typically uses the argmax. Hard EM aligns these two phases by forcing the model to pick a single winner during training.
- Sparsity: Hard EM leads to "sparser" probability distributions. Instead of distributing confidence across multiple ways to get an answer, the model learns to become highly confident in the single correct derivation.
- Qualitative Success: Manual analysis confirmed that in 92-98% of correct predictions, the model also chose the semantically correct derivation.
Conclusion
Discrete Hard EM provides a simple yet powerful alternative to MML and complex Reinforcement Learning for weakly supervised QA. By precomputing solution sets and committing to the most likely path, models can effectively navigate the noise of spurious solutions and achieve superior reasoning performance.
Summary updated