Abstract
A content selection component determines which information should be conveyed in the output of a natural language generation system. We present an efficient method for automatically learning content selection rules from a corpus and its related database. Our modeling framework treats content selection as a collective classification problem, thus allowing us to capture contextual dependencies between input items. Experiments in a sports domain demonstrate that this approach achieves a substantial improvement over context-agnostic methods.
Introduction
In concept-to-text generation, systems must sift through massive databases—such as the hundreds of statistics generated during an American football game—to decide which fraction of data is worth mentioning in a summary. Historically, this "content selection" relied on manually crafted rules that are difficult to scale. While automated machine learning approaches exist, they often treat selection as an independent "yes/no" classification for each data point, ignoring the context that makes a summary coherent. This paper introduces a collective classification approach that uses graph-based optimization to select data based on both individual importance and relationships between data points.
Methods
The proposed framework treats content selection as a collective classification problem, considering all potential data entries simultaneously rather than making isolated decisions.
Individual Preference Scores
The system first employs a standard classifier (BoosTexter) to estimate the likelihood of including a specific data entry based solely on its attributes, such as a specific player's passing yards.
Link Discovery
The system automatically discovers "links" between database entries by identifying shared attributes, such as statistics belonging to the same player or occurring in the same game quarter. A $\chi^2$ test filters out spurious links, retaining only those where the labels of connected entities are statistically dependent.
Global Optimization and Min-Cut
The selection is framed as a graph-based optimization problem. The goal is to minimize a cost function that penalizes ignoring relevant individual items and "breaking" links between related items. The authors demonstrate that this can be solved efficiently using a min-cut algorithm, which optimally partitions the graph into selected and omitted sets.
Datasets
The research utilized a domain of American Football (NFL), consisting of 468 game summaries from the official NFL website and a relational database with 73,400 entries. An automated anchor-based technique aligned database entries to summary sentences based on shared numerical values and names.
Results
The collective classifier was evaluated against a standard classifier and a majority baseline. The results show that the collective approach significantly outperforms context-agnostic methods.
As shown in the table, the collective classifier achieved an F-score of 60.15%, representing a 10.4% absolute improvement over the standard classifier (49.75%). Both precision and recall saw significant gains ($p < 0.01$), confirming that discourse constraints are vital for high-quality content selection.
Conclusion
By treating content selection as a collective task, this approach moves beyond "greedy" local decision-making. While the model currently focuses on pairwise constraints and shows moderate precision, it establishes that discourse coherence is a structural constraint that can be learned automatically. Future work includes exploring n-way constraints and integrating this component into full end-to-end generation pipelines.
Summary updated