Introduction
Summarization is a core task in natural language processing, but standard approaches often fail to capture the multi-faceted nature of complex documents. Aspect-Based Summarization (ABS) addresses this by generating summaries focused on pre-defined topics. However, in real-world scenarios, these aspects are rarely known in advance. This challenge is known as Dynamic Aspect-Based Summarization (DABS), where a model must autonomously identify the number and nature of distinct topics within a text before summarizing each individually.
Existing DABS solutions typically rely on two-step pipelines—clustering sentences followed by summarization—or prompting Large Language Models (LLMs). These methods often suffer from hallucinations, poor topic separation, or an inability to accurately count the number of underlying aspects.
Methodology: The MODABS Framework
To address these limitations, we present MODABS, a framework built upon the Longformer-Encoder-Decoder (LED) architecture. Unlike traditional models, MODABS employs a multi-objective learning framework that optimizes for three distinct goals simultaneously.
The model's innovation lies in its three loss functions:
- Cross-Entropy (Summarization Loss): Standard objective to ensure generated text matches reference summaries.
- KL Divergence (Divergence Loss): This loss penalizes similarity between summaries of different aspects. It acts as a repulsion force, preventing the model from producing generic, repetitive content across different topics.
- Aspect Classifier (Count Prediction): A dedicated head predicts the total number of aspects ($N$) present in the source text, allowing the model to adjust its output dynamically.
By modifying decoder embeddings to handle $N$ aspects in parallel, MODABS generates all specific summaries in a single computational pass.
Experimental Setup
MODABS was evaluated against three diverse datasets designed to test dynamic summarization:
- D-CnnDM: A shuffled version of the CNN/Daily Mail dataset to simulate disordered online discussions.
- D-WikiHow: A process-oriented dataset where individual steps serve as distinct aspects.
- OASUM: A large-scale open-domain dataset containing over three million samples.
Key Results and Performance
MODABS consistently outperformed both traditional baselines and state-of-the-art LLMs like GPT-3.5 across all quantitative metrics.
As shown in the results table, MODABS achieved a ROUGE-L of 26.0 on the OASUM dataset, significantly higher than the 15.8 achieved by a fine-tuned GPT-3.5. Furthermore, MODABS proved far more adept at identifying the correct number of topics. On D-CnnDM, its absolute aspect count difference (#AbsAspDiff) was a mere 1.0, whereas GPT-3.5-Zero deviated by an average of 6.2.
Human evaluation corroborated these findings, with MODABS receiving higher scores for Coherence, Consistency, Fluency, and Relevance. Evaluators specifically noted the high Aspect Quality, indicating that the model successfully isolated unique information for each topic without the bleeding common in other models.
Discussion and Conclusion
The success of MODABS highlights two critical insights for DABS: the necessity of diversity constraints (KL Divergence) to avoid the repetition trap and the importance of explicit aspect counting. While LLMs are powerful, specialized multi-objective frameworks like MODABS offer superior precision and grounding for complex, multi-topic summarization tasks.
Future work will focus on automating the balance of loss weights ($\lambda_1, \lambda_2, \lambda_3$) and reducing the hardware intensity of the multi-objective training process to make the framework more accessible for broader applications.
Summary updated