Abstract
This paper presents a novel Bidirectional Recurrent Neural Network (BRNN) architecture for factoid question answering (QA). Unlike traditional models that apply supervision only at the end of a sequence, our proposed Full-Time Supervision (FTS-BRNN) model calculates loss at every time step. This approach prevents information loss inherent in standard pooling methods and forces the network to learn robust features from every word in the prompt. Evaluated on the Quiz Bowl dataset, FTS-BRNN achieves state-of-the-art performance, significantly outperforming recursive and standard recurrent baselines.
Introduction
Factoid Question Answering (QA) requires systems to identify a specific entity based on a descriptive prompt. A common approach involves using Bidirectional Recurrent Neural Networks (BRNNs) to encode the question into a fixed-length vector, which is then compared against candidate answer representations. However, standard architectures often suffer from an "information bottleneck." By aggregating hidden states via pooling operations (e.g., max or average pooling) only at the final step, these models may discard granular semantic clues present in individual words.
Methodology
The core innovation of this work is the Full-Time Supervision based BRNN (FTS-BRNN). In this architecture, a margin loss is calculated at every single time step of the question sequence. This design ensures that the hidden state at each word is optimized to represent the correct answer, effectively treating each step as a prediction task.
The model utilizes Gated Recurrent Units (GRUs) as its primary building blocks. To bridge the gap between question hidden states and answer embeddings, a dedicated output layer is integrated on top of the BRNN. This layer maps the concatenated forward and backward hidden states into a space compatible with the answer representations. During inference, the model employs average pooling across all time steps, which functions as an ensemble of predictions from each word-level "mini-expert."
Experimental Setup
The model was evaluated using the Quiz Bowl dataset, focusing on two domains: Literature and History. The Literature subset contains 4,204 questions and 424 answers, while the History subset includes 2,557 questions and 303 answers. Data was filtered to include only answers with at least six occurrences to ensure statistical significance.
Results and Discussion
The FTS-BRNN demonstrates a substantial improvement over existing methods. As shown in the results table below, the model outperforms traditional Bag-of-Words (BOW) models, recursive neural networks like QANTA, and standard QA-LSTMs.
The transition from a standard pooling loss (83.5% on Literature) to the FTS-loss (93.1%) underscores the efficacy of dense supervision. By making every word count, the model achieves a more nuanced understanding of the input text. Furthermore, the FTS-BRNN outperformed QANTA, which is significant as recursive models require complex, pre-computed dependency parse trees, whereas the BRNN processes raw text end-to-end.
Conclusion
Full-Time Supervision provides a powerful mechanism for training BRNNs in factoid QA tasks. By alleviating the information bottleneck and leveraging the ensemble effect of word-level predictions, FTS-BRNN sets a new benchmark for accuracy. Future work will explore the applicability of this dense supervision strategy to generative and long-form question-answering tasks.
Summary updated