One-shot Learning for Question-Answering in Gaokao History Challenge

Zhuosheng Zhang, Hai Zhao

Abstract: The Gaokao history exam presents a unique challenge for artificial intelligence due to its complex semantic requirements and the scarcity of large-scale training data. This paper proposes a hybrid neural model that combines a Cooperative Gated Neural Network (CGNN) with a Neural Turing Machine (NTM). By utilizing the NTM as a one-shot labeler, the system categorizes questions into historical perspectives, allowing the CGNN to effectively retrieve accurate, lengthy answers from a textbook corpus. Experimental results demonstrate that this approach significantly outperforms traditional retrieval and classification methods in low-data scenarios.

Introduction

Most contemporary deep learning models are "data hungry," requiring vast amounts of labeled examples to achieve high performance. However, specialized domains like the Chinese university entrance exam (Gaokao) history section suffer from a severe data deficit. Unlike community question-answering (QA) platforms, there is no massive repository of labeled history QA pairs.

Furthermore, Gaokao history questions are rarely simple factoid lookups. They are "comprehensive" questions requiring an understanding of historical perspectives, such as the causes of events or the influence of movements. Answers are typically "detail-riddled" paragraphs rather than short sentences. Standard models often fail to filter the noise in these long sequences and struggle to generalize from the limited number of available past exam questions.

Methodology

The researchers developed a two-part hybrid architecture designed to mimic human conceptual learning and selective attention.

Neural Turing Machine (NTM) Labeler

To address the lack of data, the model employs an NTM as a "one-shot" classifier. Inspired by human cognitive processes, the NTM uses an external memory bank to store and retrieve information rapidly. This module classifies questions into five key historical perspectives: Background, Cause, Claim, Fact, and Influence. This classification narrows the search space for the retrieval engine.

Cooperative Gated Neural Network (CGNN)

The CGNN serves as the core retrieval engine, taking the question (enriched with the NTM's label) and candidate answers from a textbook corpus. A critical component of the CGNN is its "Gated Flow" mechanism. Drawing inspiration from Highway Networks, the CGNN uses input, forget, and output gates to control information flow during convolution. This allows the model to "ignore" irrelevant words in lengthy, noisy answers and focus on salient semantic features.

The hybrid architecture (left) and the detailed Gated Flow mechanism within the CGNN (right).

Experimental Setup

The authors introduced a new benchmark for this task, the Gaokao History Corpus, compiled from standard Chinese history textbooks and past exam papers. The dataset includes 1,929 annotated QA pairs and a candidate answer set of 1,929 text fragments extracted by human experts. For the one-shot learning component, a subset of 400 samples (80 per class) was used to train the NTM labeler in scenarios ranging from 1-shot to 10-shot.

Results and Discussion

The NTM’s ability to "rapidly bind" information into external memory allowed it to achieve 76.6% accuracy in a 10-shot setting, nearly doubling the performance of traditional classifiers like SVMs and significantly outperforming standard LSTMs (38.2%).

Classification accuracy of the NTM labeler over training steps for 1, 2, 5, and 10-shot learning.

In the final answer retrieval task, the full model (CGNN + NTM) achieved an F1-score of 40.8. This represents a massive improvement over standard BM25 search engines (12.7) and the CGNN without NTM labels (23.1). The gating mechanism proved essential for handling the long, detail-heavy answers characteristic of history exams, acting as a filter that assigns higher weights to key thematic words like "productivity" or "influence."

Performance varied by question type; the model excelled at "Cause" and "Claim" questions (F1 ~55-56) but struggled with "Fact" questions (F1 ~29.7). This suggests that factual questions are more diverse and harder to distinguish from other categories using this specific labeling system.

Conclusion

This research demonstrates that combining external memory modules with gated neural architectures provides a robust path forward for QA in data-scarce, complex domains. While the model excels in low-data environments, future work is needed to handle entirely unstructured textbook data and to improve reasoning for highly diverse "Fact" type questions. The release of the Gaokao History Corpus aims to encourage further community research into comprehensive, perspective-driven question answering.

Summary updated

BibTeX
@inproceedings{zhang2018one,
  author = {Zhuosheng Zhang and Hai Zhao},
  title = {{One-shot Learning for Question-Answering in Gaokao History Challenge}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
  url = {https://openreview.net/forum?id=HfW7cYh5RY},
}
Figure