MODABS: Multi-Objective Learning for Dynamic Aspect-Based Summarization

Xiaobo Guo, Soroush Vosoughi

Introduction

Summarization is a core task in natural language processing, but standard approaches often fail to capture the multi-faceted nature of complex documents. Aspect-Based Summarization (ABS) addresses this by generating summaries focused on pre-defined topics. However, in real-world scenarios, these aspects are rarely known in advance. This challenge is known as Dynamic Aspect-Based Summarization (DABS), where a model must autonomously identify the number and nature of distinct topics within a text before summarizing each individually.

Existing DABS solutions typically rely on two-step pipelines—clustering sentences followed by summarization—or prompting Large Language Models (LLMs). These methods often suffer from hallucinations, poor topic separation, or an inability to accurately count the number of underlying aspects.

Methodology: The MODABS Framework

To address these limitations, we present MODABS, a framework built upon the Longformer-Encoder-Decoder (LED) architecture. Unlike traditional models, MODABS employs a multi-objective learning framework that optimizes for three distinct goals simultaneously.

The MODABS architecture integrates aspect-based embeddings with a multi-objective loss function to ensure diversity and accuracy.

The model's innovation lies in its three loss functions:

  1. Cross-Entropy (Summarization Loss): Standard objective to ensure generated text matches reference summaries.
  2. KL Divergence (Divergence Loss): This loss penalizes similarity between summaries of different aspects. It acts as a repulsion force, preventing the model from producing generic, repetitive content across different topics.
  3. Aspect Classifier (Count Prediction): A dedicated head predicts the total number of aspects ($N$) present in the source text, allowing the model to adjust its output dynamically.

By modifying decoder embeddings to handle $N$ aspects in parallel, MODABS generates all specific summaries in a single computational pass.

Experimental Setup

MODABS was evaluated against three diverse datasets designed to test dynamic summarization:

  • D-CnnDM: A shuffled version of the CNN/Daily Mail dataset to simulate disordered online discussions.
  • D-WikiHow: A process-oriented dataset where individual steps serve as distinct aspects.
  • OASUM: A large-scale open-domain dataset containing over three million samples.

Key Results and Performance

MODABS consistently outperformed both traditional baselines and state-of-the-art LLMs like GPT-3.5 across all quantitative metrics.

MODABS demonstrates superior performance in ROUGE, BERTScore, and aspect count accuracy (#AbsAspDiff) compared to keywords, clustering, and GPT-3.5 baselines.

As shown in the results table, MODABS achieved a ROUGE-L of 26.0 on the OASUM dataset, significantly higher than the 15.8 achieved by a fine-tuned GPT-3.5. Furthermore, MODABS proved far more adept at identifying the correct number of topics. On D-CnnDM, its absolute aspect count difference (#AbsAspDiff) was a mere 1.0, whereas GPT-3.5-Zero deviated by an average of 6.2.

Human evaluation corroborated these findings, with MODABS receiving higher scores for Coherence, Consistency, Fluency, and Relevance. Evaluators specifically noted the high Aspect Quality, indicating that the model successfully isolated unique information for each topic without the bleeding common in other models.

Discussion and Conclusion

The success of MODABS highlights two critical insights for DABS: the necessity of diversity constraints (KL Divergence) to avoid the repetition trap and the importance of explicit aspect counting. While LLMs are powerful, specialized multi-objective frameworks like MODABS offer superior precision and grounding for complex, multi-topic summarization tasks.

Future work will focus on automating the balance of loss weights ($\lambda_1, \lambda_2, \lambda_3$) and reducing the hardware intensity of the multi-objective training process to make the framework more accessible for broader applications.

Summary updated

Abstract

The rapid proliferation of online content necessitates effective summarization methods, among which dynamic aspect-based summarization stands out. Unlike its traditional counterpart, which assumes a fixed set of known aspects, this approach adapts to the varied aspects of the input text. We introduce a novel multi-objective learning framework employing a Longformer-Encoder-Decoder for this task. The framework optimizes aspect number prediction, minimizes disparity between generated and reference summaries for each aspect, and maximizes dissimilarity across aspect-specific summaries. Extensive experiments show our method significantly outperforms baselines on three diverse datasets, largely due to the effective alignment of generated and reference aspect counts without sacrificing single-aspect summarization quality.
BibTeX
@inproceedings{guo2024modabs,
  author = {Xiaobo Guo and Soroush Vosoughi},
  title = {{MODABS: Multi-Objective Learning for Dynamic Aspect-Based Summarization}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.findings-acl.165},
}
Figure