Abstract
Summarizing health and nutrition studies requires high faithfulness and relevance, which standard sequence-to-sequence models often fail to provide due to hallucinations and data scarcity. We present Nutri-bullets, an "extract-compose" framework that explicitly extracts factual spans and uses reinforcement learning to select and fuse them. Our approach significantly outperforms baselines in faithfulness and relevance on two new health summarization benchmarks, particularly in low-resource settings.
Introduction
In the health and nutrition domain, the rapid publication of scientific studies necessitates efficient summarization for public consumption. However, manual summarization is resource-intensive and requires significant domain expertise. While modern AI models excel at generating fluent text, they frequently suffer from two major issues in high-stakes fields: unfaithfulness (hallucinating facts) and data scarcity (lack of high-quality training pairs). Standard sequence-to-sequence (seq2seq) models often fail to accurately capture essential facts from long, multi-document inputs, leading to summaries that may be fluent but factually incorrect.
Methods: The Extract-Compose Pipeline
Nutri-bullets addresses these challenges through a structured three-stage pipeline that prioritizes factual grounding over unconstrained generation.
- Span Extraction ($P(Z|X)$): Instead of processing the entire text at once, the model identifies specific "knowledge spans"—short phrases containing critical entity-relation information (e.g., "Flaxseed extract may improve heart health").
- Content Selection Policy ($P(C|Z)$): A Reinforcement Learning (RL) policy network treats selection as a sequential decision process, picking an ordered subset of spans from a pool. The RL agent is rewarded based on how well the selection mirrors the structure of human-written summaries and ensures information diversity.
- Text Fusion ($P(y|C)$): A Blank Language Model (BLM) fills the gaps between selected spans to create a fluent sentence. This model is specifically designed to ensure the final output is readable without altering the underlying facts.
Datasets and Evaluation
The framework was evaluated on two new benchmarks introduced by the authors:
- HealthLine: 1,894 instances of scientific abstracts paired with short, bulleted summaries.
- BreastCancer: A smaller, more challenging dataset of 141 instances where summaries reference an average of 18 articles each.
To train the extraction modules, the authors also collected a massive dataset of 91K entities and 34K relation tuples (e.g., "Food X" -> "Increases" -> "Condition Y").
Results
Nutri-bullets, particularly the variant using the Blank Language Model (BLM), demonstrated superior performance across multiple metrics compared to standard Transformer and GraphWriter baselines.
Human evaluators rated Nutri-bullets significantly higher for faithfulness and relevance. On the challenging BreastCancer dataset, Nutri-bullets was preferred 94% of the time over a strong Transformer baseline. The model also achieved higher Knowledge Graph (KG) metrics, proving it captures actual scientific relationships better than standard seq2seq models. While standard models often achieve high fluency, they frequently fail on faithfulness; Nutri-bullets effectively "anchors" the AI to the truth.
Conclusion
By shifting from "black box" generation to a structured "extract-compose" framework, Nutri-bullets provides a more reliable method for summarizing technical health data. While the model's performance depends on the quality of initial span extraction, it represents a significant step forward in generating faithful summaries for high-stakes domains. Future work will focus on improving sentence cohesion and scaling the approach to broader scientific fields.
Summary updated