What If Sentence-hood is Hard to Define: A Case Study in Chinese Reading Comprehension

Jiawei Wang, Hai Zhao, Yinggong Zhao, Libin ShenEMNLP 2021 · DOI

Abstract

Extractive Machine Reading Comprehension (MRC) typically focuses on token-level span identification. However, languages like Chinese present a unique challenge known as "location unit ambiguity," where sentence boundaries are semantically fluid rather than syntactically rigid. This paper introduces ESPReader, a framework that explicitly predicts answer-containing sentences through a multi-task learning approach. By incorporating sentence-level supervision, ESPReader achieves state-of-the-art results on Chinese benchmarks and demonstrates robust performance across English, Japanese, and French datasets.

Introduction: The Challenge of Location Unit Ambiguity

In extractive MRC, models are tasked with identifying a specific text span within a passage to answer a query. While English models benefit from clear syntactic markers (like periods) that define sentence boundaries, Chinese "sentences" are often semantic clusters where sub-sentences are separated by commas while sharing a single subject.

This lack of clear boundaries leads to "location unit ambiguity." Models often struggle to determine the appropriate context, resulting in "omitting errors" (spans that are too short) or "surplus errors" (spans that include irrelevant text). Humans naturally use a hierarchical approach—locating the relevant sentence before zooming into the specific answer—a strategy that standard token-level models lack.

Comparison of sentence-level evidence locating between Chinese and English readers, highlighting the ambiguity in Chinese sentence boundaries.

Methodology: The ESPReader Framework

To address these issues, we propose ESPReader (Reader with Explicit Span-sentence Predication). The model shifts from a purely token-based approach to a hierarchical one that explicitly models sentence boundaries.

1. Explicit Span-sentence Predication (ESP)

During training, the model is assigned a secondary task: identifying which "natural sentence" (segmented by punctuation) contains the ground-truth answer. This forces the model to learn the boundaries of evidence units explicitly.

2. Hierarchical Architecture

The ESPReader architecture integrates sentence-level information into the token-level extraction process through several key components:

  • Sentence-level Representation: Token embeddings from a Pre-trained Language Model (PrLM) are aggregated to form sentence vectors.
  • Sentence-level Self-attention (SSA): This layer captures global context by allowing different sentences to interact and understand the broader passage structure.
  • Fusion Cross-attention (FCA): High-level sentence features are fused back into token representations, guiding the model to focus its search within the predicted evidence unit.
The hierarchical architecture of ESPReader, featuring the SSA and FCA layers for integrating sentence-level context.

3. Inverse Length Factor (ILF)

To prevent the model from simply selecting entire long sentences, an Inverse Length Factor (ILF) is applied. This scoring penalty discourages predicted spans that deviate significantly from the expected answer length in the dataset.

Experimental Evaluation

ESPReader was evaluated across multiple benchmarks, including CMRC 2018 (Simplified Chinese), DRCD (Traditional Chinese), and SQuAD 2.0 (English).

Key Findings

  • State-of-the-Art Performance: ESPReader set new benchmarks on the CMRC 2018 leaderboard for base-level models.
  • Model Efficiency: Remarkably, the MacBERT-base version of ESPReader outperformed the standard MacBERT-large model in Exact Match (EM) scores, demonstrating that structural improvements can be more effective than simply increasing parameter count.
  • Cross-Lingual Generalization: Despite being motivated by Chinese linguistic features, the model improved performance on Japanese MRC by nearly 8% and French by 1.8%. It also showed gains on the English SQuAD dataset, suggesting that identifying evidence units is a universal challenge.
  • Error Reduction: Analysis showed a 26% reduction in "total miss" errors (F1=0), indicating that the model is significantly better at staying within the correct evidence region.

Conclusion

The success of ESPReader underscores the importance of considering linguistic relativity in NLP architecture. By explicitly modeling sentence-hood—a concept often taken for granted in English-centric models—we can build more robust reading comprehension systems. Future work will explore applying this explicit location logic to multi-hop reasoning tasks where evidence is scattered across multiple sentences.

Summary updated

BibTeX
@inproceedings{wang2021what,
  author = {Jiawei Wang and Hai Zhao and Yinggong Zhao and Libin Shen},
  title = {{What If Sentence-hood is Hard to Define: A Case Study in Chinese Reading Comprehension}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.findings-emnlp.202},
}
Figure