LTRC-IIITH at PerAnsSumm 2025: SpanSense - Perspective-specific span identification and Summarization

Sushvin Marimuthu, Parameswari KrishnamurthyNAACL 2025

TL;DR

This paper tackles the challenge of summarizing medical forum answers by identifying and classifying specific "perspective" spans within the text, then generating summaries tailored to those perspectives. Their system, "SpanSense," uses RoBERTa-large for span identification and BART-large for summarization, achieving a competitive 49% overall score and ranking 6th in the PerAnsSumm 2025 shared task.

What Problem Does This Paper Address?

Healthcare question-answering (CQA) forums are a valuable resource for users seeking medical advice. However, traditional summarization methods often rely on the single "best-voted" answer, overlooking the diverse perspectives and nuances present across multiple responses. This can lead to incomplete or biased information. The PerAnsSumm shared task aims to address this by developing systems that can: 1) identify and classify specific spans of text that represent distinct perspectives (e.g., "cause," "suggestion," "experience"), and 2) generate summaries that capture these individual perspectives from entire question-answer threads.

Methods

The SpanSense system employs a two-stage approach:

  • Task A: Perspective-Specific Span Identification: The authors fine-tuned pre-trained language models, specifically BERT-large and RoBERTa-large, for this task. They experimented with different token classification formats (BIO, IO, BIOES) and found RoBERTa-large to be superior. To further enhance performance, they utilized domain-adaptive pre-training using Masked Language Modeling (MLM) and a novel "gradual training" technique. This gradual training involved progressively unfreezing layers of the model over several epochs, allowing for more nuanced learning.
  • Task B: Perspective-Specific Summarization: For generating summaries from the identified spans, the authors fine-tuned BART-large and Pegasus-large models. Similar to Task A, domain-adaptive pre-training with MLM was applied to improve performance. BART-large ultimately outperformed Pegasus-large.

Datasets

The primary dataset used is the PUMA dataset, specifically designed for the PerAnsSumm 2025 shared task. This dataset consists of medical CQA threads, with answers annotated for five perspective spans: 'cause', 'suggestion', 'experience', 'question', and 'information'. Each identified span is also associated with a concise summary. The dataset was split into 2,236 training, 959 validation, and 50 testing instances.

Key Insights and Ideas

  • The Power of Perspective: The core idea is that simply summarizing the "best" answer is insufficient. Extracting and summarizing distinct perspectives (like advice vs. personal experience) provides a richer, more nuanced understanding for users.
  • RoBERTa-large Shines for Span Identification: RoBERTa-large, especially after domain-adaptive pre-training and gradual fine-tuning, proved to be the most effective model for accurately identifying and classifying perspective spans.
  • Gradual Training is Key: The "gradual training" approach, where model layers are unfrozen incrementally, significantly boosted the performance of RoBERTa-large for span identification, highlighting its effectiveness in fine-tuning complex models.
  • BART-large for Coherent Summaries: BART-large demonstrated strong performance in generating perspective-specific summaries, effectively condensing complex information while retaining the essence of each viewpoint.

Key Results

  • Task A (Span Identification): The final RoBERTa-large model achieved a Macro-F1 score of 0.9 (90%) and a Weighted-F1 score of 0.92 (92%) for classification. For span matching, strict matching F1 was 0.21 (21%) and proportional matching was 0.68 (68%), leading to an average Task A score of 0.6 (60%).
  • Task B (Summarization): The BART-large model achieved a ROUGE-1 score of 0.4 (40%), ROUGE-2 of 0.18 (18%), and ROUGE-L of 0.36 (36%). Other metrics included BERTScore of 0.84 (84%) and METEOR of 0.37 (37%), resulting in an average Task B score of 0.38 (38%).
  • Overall Performance: Combining both tasks, the system achieved an overall average score of 49% and ranked 6th on the official leaderboard.

Limitations and Future Work

The authors acknowledge several limitations:

  • Data Dependency: The system relies heavily on manually annotated training data, which can be costly and time-consuming to acquire, posing a challenge for scalability to new domains.
  • Overlapping/Implicit Perspectives: Handling spans with multiple viewpoints or perspectives that are implied rather than explicitly stated remains a challenge, potentially leading to incomplete summaries.
  • Factual Accuracy and Neutrality: The current approach focuses on extracting and summarizing perspective-specific information but does not verify factual accuracy or neutrality, which could be critical for real-world medical applications.

Future work could explore:

  • Optimized Training Strategies: Investigating alternative training methods beyond gradual training.
  • Ambiguity Handling: Developing better mechanisms to handle ambiguous perspectives.
  • Fact-Checking Integration: Incorporating fact-checking mechanisms to ensure the reliability of generated summaries.

Summary updated

BibTeX
@inproceedings{marimuthu2025ltrc,
  author = {Sushvin Marimuthu and Parameswari Krishnamurthy},
  title = {{LTRC-IIITH at PerAnsSumm 2025: SpanSense - Perspective-specific span identification and Summarization}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure