Abstract
Extractive Machine Reading Comprehension (MRC) typically focuses on token-level span identification. However, languages like Chinese present a unique challenge known as "location unit ambiguity," where sentence boundaries are semantically fluid rather than syntactically rigid. This paper introduces ESPReader, a framework that explicitly predicts answer-containing sentences through a multi-task learning approach. By incorporating sentence-level supervision, ESPReader achieves state-of-the-art results on Chinese benchmarks and demonstrates robust performance across English, Japanese, and French datasets.
Introduction: The Challenge of Location Unit Ambiguity
In extractive MRC, models are tasked with identifying a specific text span within a passage to answer a query. While English models benefit from clear syntactic markers (like periods) that define sentence boundaries, Chinese "sentences" are often semantic clusters where sub-sentences are separated by commas while sharing a single subject.
This lack of clear boundaries leads to "location unit ambiguity." Models often struggle to determine the appropriate context, resulting in "omitting errors" (spans that are too short) or "surplus errors" (spans that include irrelevant text). Humans naturally use a hierarchical approach—locating the relevant sentence before zooming into the specific answer—a strategy that standard token-level models lack.
Methodology: The ESPReader Framework
To address these issues, we propose ESPReader (Reader with Explicit Span-sentence Predication). The model shifts from a purely token-based approach to a hierarchical one that explicitly models sentence boundaries.
1. Explicit Span-sentence Predication (ESP)
During training, the model is assigned a secondary task: identifying which "natural sentence" (segmented by punctuation) contains the ground-truth answer. This forces the model to learn the boundaries of evidence units explicitly.
2. Hierarchical Architecture
The ESPReader architecture integrates sentence-level information into the token-level extraction process through several key components:
- Sentence-level Representation: Token embeddings from a Pre-trained Language Model (PrLM) are aggregated to form sentence vectors.
- Sentence-level Self-attention (SSA): This layer captures global context by allowing different sentences to interact and understand the broader passage structure.
- Fusion Cross-attention (FCA): High-level sentence features are fused back into token representations, guiding the model to focus its search within the predicted evidence unit.
3. Inverse Length Factor (ILF)
To prevent the model from simply selecting entire long sentences, an Inverse Length Factor (ILF) is applied. This scoring penalty discourages predicted spans that deviate significantly from the expected answer length in the dataset.
Experimental Evaluation
ESPReader was evaluated across multiple benchmarks, including CMRC 2018 (Simplified Chinese), DRCD (Traditional Chinese), and SQuAD 2.0 (English).
Key Findings
- State-of-the-Art Performance: ESPReader set new benchmarks on the CMRC 2018 leaderboard for base-level models.
- Model Efficiency: Remarkably, the MacBERT-base version of ESPReader outperformed the standard MacBERT-large model in Exact Match (EM) scores, demonstrating that structural improvements can be more effective than simply increasing parameter count.
- Cross-Lingual Generalization: Despite being motivated by Chinese linguistic features, the model improved performance on Japanese MRC by nearly 8% and French by 1.8%. It also showed gains on the English SQuAD dataset, suggesting that identifying evidence units is a universal challenge.
- Error Reduction: Analysis showed a 26% reduction in "total miss" errors (F1=0), indicating that the model is significantly better at staying within the correct evidence region.
Conclusion
The success of ESPReader underscores the importance of considering linguistic relativity in NLP architecture. By explicitly modeling sentence-hood—a concept often taken for granted in English-centric models—we can build more robust reading comprehension systems. Future work will explore applying this explicit location logic to multi-hop reasoning tasks where evidence is scattered across multiple sentences.
Summary updated