Classification-Aware Neural Topic Model Combined with Interpretable Analysis - for Conflict Classification

Tianyu Liang, Yida Mu, Soonho Kim, Darline Kengne Kuate, Julie Lang, Rob Vos, Xingyi Song

Introduction

In the wake of global crises, international organizations like the IFPRI require rapid classification and analysis of conflict information to coordinate humanitarian responses. While deep learning models excel at classification, they often function as "black boxes," making it difficult to discern why a specific label—such as "riot" versus "battle"—was assigned. Furthermore, traditional topic models frequently struggle with "neutral" noise words (e.g., "report" or "city") that do not help distinguish conflict types.

To address these challenges, researchers developed CANTM-IA, a neural topic model that simultaneously classifies conflict events and discovers related topics while providing human-readable explanations. By focusing on "rationales"—the most important words identified via saliency scores—the model filters out noise and achieves state-of-the-art accuracy.

The CANTM-IA Framework

The Classification-Aware Neural Topic Model with Interpretable Analysis (CANTM-IA) builds upon the CANTM framework with several key innovations designed to enhance both performance and interpretability.

The architecture of the CANTM-IA model, highlighting the integration of BERT embeddings with saliency-based rationale reconstruction.
  1. Saliency-Based Rationales: The model utilizes attention weights from a pre-trained BERT layer to calculate saliency scores. These scores identify specific words (rationales) that most heavily influence the classification decision.
  2. Targeted Reconstruction: Unlike standard Variational Auto-Encoders (VAEs) that attempt to reconstruct the entire document, CANTM-IA focuses its reconstruction efforts on high-saliency rationales. This forces the latent topic space to ignore irrelevant noise and focus on core conflict information.
  3. Architecture Optimization: Redundant structures in the original CANTM decoder were removed to reduce computational complexity without sacrificing performance.
  4. Hybrid Learning: The model combines BERT’s contextual embeddings with a stacked VAE architecture to discover latent topics effectively.

Experimental Setup

The study utilized a large-scale dataset of over 522,000 items from two primary sources:

  • ACLED (Armed Conflict Location & Event Data Project): Covering six event types (Battles, Explosions, Protests, Riots, Strategic Developments, and Violence Against Civilians) from 2019 to 2022.
  • WMT News Crawl: Used as out-of-domain data to test the model's ability to distinguish conflict events from general news.

Results and Analysis

CANTM-IA outperformed both the standard BERT baseline and the original CANTM model across all primary metrics. Notably, the best-performing version (CANTM-IA ratio 0.5) achieved an accuracy of 0.9780 and an F1 score of 0.9791.

Comparative performance of BERT, CANTM, and CANTM-IA variants showing improvements in Accuracy and F-1 scores.

Beyond quantitative gains, the model showed significant qualitative improvements. By reconstructing rationales instead of full text, the model successfully filtered out neutral words like "unknown" or "reported," resulting in cleaner, more descriptive topic lists. For example, in the "Battles" category, the model prioritized active terms like "clashed" and "attacked" over generic terms like "forces."

Conclusion

CANTM-IA demonstrates that interpretability does not have to come at the cost of performance; rather, using saliency scores to guide a model's focus can actually improve classification accuracy. Future work will focus on testing the model's robustness on noisier, raw social media data and refining the precision of the generated rationales to further enhance transparency in high-stakes humanitarian decision-making.

Summary updated

Abstract

A large number of conflict events are affecting the world all the time. In order to analyse such conflict events effectively, this paper presents a Classification-Aware Neural Topic Model (CANTM-IA) for Conflict Information Classification and Topic Discovery. The model provides a reliable interpretation of classification results and discovered topics by introducing interpretability analysis. At the same time, interpretation is introduced into the model architecture to improve the classification performance of the model and to allow interpretation to focus further on the details of the data. Finally, the model architecture is optimised to reduce the complexity of the model.
BibTeX
@inproceedings{liang2023classification,
  author = {Tianyu Liang and Yida Mu and Soonho Kim and Darline Kengne Kuate and Julie Lang and Rob Vos and Xingyi Song},
  title = {{Classification-Aware Neural Topic Model Combined with Interpretable Analysis - for Conflict Classification}},
  booktitle = {RANLP},
  year = {2023},
}
Figure