Abstract: Hojae Han, Seungtaek Choi, Haeju Park, Seung-won Hwang. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).
2019.
Introduction
In the field of Question Answering (QA), researchers distinguish between "factoid" questions, which seek short, specific entities (e.g., "What is the capital of France?"), and "non-factoid" questions, which require long-form, multi-sentence explanations (e.g., "How does the education process work?"). While neural models have excelled at the former, non-factoid QA presents a unique challenge: candidate answers are often long passages where the relevant information is buried in context.
Existing neural approaches typically fall into two categories. Representation-focused models encode questions and answers into fixed-length vectors for fast comparison, but they often lose granular detail in long passages. Interaction-focused models build a matrix of matches between every word or n-gram in the question and answer, preserving structural detail but struggling to incorporate traditional information retrieval signals like query term weighting.
The MICRON Architecture
To bridge this gap, the authors propose MICRON (Multigranular Interaction for Contextualizing RepresentatiON). MICRON is a hybrid architecture designed to combine the efficiency of representation-based models with the structural precision of interaction-based models.
The architecture consists of three primary modules:
- Encoding Module: This layer uses n-gram CNNs with varying window sizes (e.g., 1, 2, 3, and 5 words) to capture local semantics at different granularities.
- Contextualization Layer ($\Phi$): This core innovation takes the various n-gram representations and averages them back down to the word level. This creates a "contextualized" word vector that retains information about its neighbors, allowing the model to distinguish between different uses of the same word based on its surrounding phrase.
- Matching & Scoring: The model computes a dot-product interaction matrix between the contextualized question and answer words. Because the final representation is word-based, the model can easily integrate Inverse Document Frequency (IDF) signals to prioritize rare, meaningful query terms.
Key Innovation: Contextualized Word Signals
A significant advantage of MICRON is its ability to perform M:N matching—matching a short phrase in a question to a longer, semantically similar phrase in an answer. However, unlike pure interaction models, MICRON remains "word-aware."
By mapping multi-granular interactions back to the word level, the model can apply traditional weighting signals. As shown in the qualitative analysis, the model can effectively "down-weight" common matches (like the word "is") while amplifying the signal for high-information terms (like "Sweden"). This ensures that the deep learning process is guided by proven information retrieval principles.
Experimental Results
MICRON was evaluated on two major non-factoid QA benchmarks: WikiPassageQA and InsuranceQA.
- InsuranceQA: MICRON achieved an accuracy of 49.8%, a significant improvement over the standard N-gram CNN baseline (45.8%).
- WikiPassageQA: On the Mean Average Precision (MAP) metric, MICRON reached 63.00, outperforming both the BM25 baseline (53.73) and several complex interaction-focused models.
Beyond accuracy, the authors emphasize computational efficiency. With approximately 1 million parameters, MICRON is significantly more energy-efficient and faster to deploy than massive transformer models like BERT (110M+ parameters), while remaining competitive on specific retrieval-heavy tasks.
Conclusion
MICRON demonstrates that contextualizing word-level representations with multi-granular interactions provides a powerful and efficient framework for non-factoid QA. By allowing "old school" signals like IDF to guide modern neural architectures, MICRON achieves state-of-the-art performance without the massive computational overhead of larger models. Future work may involve integrating these multi-granular interaction layers into the attention mechanisms of Transformer-based architectures.
Summary updated