Detecting Short-lasting Topics Using Nonnegative Tensor Decomposition.

Lara Kassab, Alona Kryshchenko, Hanbaek Lyu, Denali Molitor, Deanna Needell, Elizaveta Rebrova

Abstract

Temporal data, such as news articles or Twitter feeds, often consists of a mixture of long-lasting trends and popular but short-lasting topics of interest. A truly successful topic modeling strategy should be able to detect both types of topics and clearly locate them in time. In this paper, we compare the variability of topic lengths discovered by several well-known topic modeling methods including latent Dirichlet allocation (LDA), nonnegative matrix factorization (NMF), as well as its tensor counterparts based on the nonnegative CANDECOMP/PARAFAC tensor decomposition (NCPD and Online NCPD). We demonstrate that only tensor-based methods with the dedicated mode for tracking time evolution successfully detect short-lasting topics. Furthermore, these methods are considerably more accurate in discovering the points in time when topics appeared and disappeared compared to the matrix-based methods such as LDA and NMF. We propose quantitative ways to measure the topic length and demonstrate the ability of NCPD (as well as its online variant), to discover short and long-lasting temporal topics in semi-synthetic and real-world data including news headlines and COVID-19 related tweets.

Introduction

Dynamic topic modeling aims to discover how themes evolve over time in massive datasets like news feeds or social media. However, traditional methods like Latent Dirichlet Allocation (LDA) and Nonnegative Matrix Factorization (NMF) often suffer from a "persistence bias." These models are excellent at identifying long-lasting trends (e.g., "Politics" or "Sports") but frequently fail to capture short-lived, impactful events (e.g., a specific natural disaster or a viral news story). This occurs because the signals of fleeting events are often "washed out" when temporal data is collapsed into 2D matrices.

To address this, we propose a move from matrix-based models to Sparseness-constrained Nonnegative Tensor Factorization (S-NCPD). By utilizing a 3D tensor representation—Words × Documents × Time—we can preserve the multi-way structure of the data and accurately isolate topics across varying time scales.

Methodology

The core of our approach is the use of Nonnegative CANDECOMP/PARAFAC decomposition (NCPD) on a 3D data tensor. Unlike NMF, which blends subjects if they appear in the same general timeframe, NCPD treats the document-batch and the time-slice as separate modes.

S-NCPD and OS-NCPD

We introduce two primary innovations to the tensor framework:

  1. S-NCPD (Sparseness-constrained NCPD): We enforce a tunable sparseness constraint on the "time × topic" factor matrix using Hoyer’s sparseness measure. This acts as a "control knob," allowing researchers to force the model to find sparse (short-lived) topics or allow for dense (long-lasting) trends.
  2. OS-NCPD (Online S-NCPD): To handle massive datasets that exceed memory limits, we developed an online version that processes data in mini-batches, progressively updating topic and time factors.

Quantitative Metrics

To mathematically define topic duration, we introduce the α-effective length (the minimum time units needed to account for a certain percentage of a topic's "mass") and normalized AUC (nAUC). These metrics allow for a principled comparison of how different models handle temporal localization.

Experimental Evaluation

We evaluated the models using three distinct datasets:

  • Semi-synthetic 20 Newsgroups: A controlled dataset where specific categories (like "Atheism" or "Space") were manually placed into specific time slices to test recovery accuracy.
  • ABC News Headlines: Over 142,000 headlines from 2003–2019, organized into a 203 × 7000 × 700 tensor.
  • COVID-19 Tweets: Real-world social media data used to track rapidly evolving public discourse.

Key Results

Our findings demonstrate several advantages of tensor-based decomposition over traditional methods:

  • Superior Localization: In the 20 Newsgroups experiment, NCPD and OS-NCPD accurately identified the exact time slices for short-lived topics. In contrast, NMF and LDA "smeared" these topics across adjacent time periods, failing to identify clear temporal boundaries.
  • Topic Diversity: On the ABC News dataset, LDA and NMF mostly discovered topics with high persistence (nAUC > 0.9). S-NCPD successfully identified a mix of scales, including "fleeting" topics with nAUC scores as low as 0.4.
  • Efficiency and Tunability: The online variant (OS-NCPD) reduced reconstruction error significantly faster than the batch version. Furthermore, we observed a nearly linear relationship between the input sparseness parameter and the resulting average topic length, proving the model can be effectively "steered" by the user.

Conclusion and Future Work

By moving to a tensor-based framework, we can overcome the persistence bias inherent in matrix-based topic modeling. S-NCPD provides a robust way to detect both short-lasting and long-lasting topics with high temporal precision. Future work will investigate automated rank selection and the application of these localization properties to other fields, such as spatial localization in image analysis or short-term gene expression in computational biology.

Summary updated

Abstract

Temporal data (such as news articles or Twitter feeds) often consists of a mixture of long-lasting trends and popular but short-lasting topics of interest. A truly successful topic modeling strategy should be able to detect both types of topics and clearly locate them in time. In this paper, we compare the variability of topic lengths discovered by several well-known topic modeling methods including latent Dirichlet allocation (LDA), nonnegative matrix factorization (NMF), as well as its tensor counterparts based on the nonnegative CANDECOMP/PARAFAC tensor decomposition (NCPD and Online NCPD). We demonstrate that only tensor-based methods with the dedicated mode for tracking time evolution successfully detect short-lasting topics. Furthermore, these methods are considerably more accurate in discovering the points in time when topics appeared and disappeared compared to the matrix-based methods such as LDA and NMF. We propose quantitative ways to measure the topic length and demonstrate the ability of NCPD (as well as its online variant), to discover short and long-lasting temporal topics in semi-synthetic and real-world data including news headlines and COVID-19 related tweets.
BibTeX
@article{kassab2024detecting,
  author = {Lara Kassab and Alona Kryshchenko and Hanbaek Lyu and Denali Molitor and Deanna Needell and Elizaveta Rebrova},
  title = {{Detecting Short-lasting Topics Using Nonnegative Tensor Decomposition.}},
  journal = {Frontiers Appl. Math. Stat.},
  year = {2024},
  doi = {10.3389/fams.2024.1287074},
}
Figure