Interpretable Neural Networks via Semantic Clustering

16 concepts13 papers1 proposal

Deep neural networks usually represent data as dense, continuous vectors in a high-dimensional space. While these embeddings are exceptionally good at capturing complex patterns, they are notoriously uninterpretable. If a model classifies an image as a "cat" or an SMS as "spam," the underlying reasoning is buried in millions of floating-point operations.

To resolve this, a prominent research direction introduces a discrete bottleneck into neural architectures: semantic clustering. Instead of passing raw, uninterpretable vectors directly to a decision layer, these architectures force the model to group data—whether words in a document or patches in an image—into discrete, semantically meaningful clusters or topics. Because these clusters map directly to human-understandable concepts (like a collection of related words or a distinct object part), the model's internal reasoning becomes transparent. We can see exactly which clusters are driving the final prediction.

Text Modality: Grounding Decisions in Latent Topics

In natural language processing, semantic clustering often takes the form of topic modeling. Traditional topic models like Latent Dirichlet Allocation (LDA) or Nonnegative Matrix Factorization (NMF) group words into thematic clusters, but they lack the deep contextual awareness of modern transformers. The key insight of recent hybrid architectures is to combine them: use deep networks to capture context, but route their representations through an interpretable topic-modeling bottleneck.

For instance, Noura Al Moubayed et al. combined LDA with Stacked Denoising Autoencoders in SMS Spam Filtering using Probabilistic Topic Modelling and Stacked Denoising Autoencoder, demonstrating that visualizing the word clusters driving the model's decisions makes anomaly detection transparent. Similarly, in Semi-supervised Nonnegative Matrix Factorization for Document Classification, Jamie Haddock et al. utilized semi-supervised NMF to simultaneously perform classification and reveal the latent topics that justify those categories, eliminating the black-box nature of document classification.

Why does this hybrid approach work so well? By forcing the network to express its internal state as a distribution over explicit topics, predictions become mathematically tied to observable thematic spikes. For example, in the Classification-Aware Neural Topic Model (CANTM) proposed by Xingyi Song et al. in Classification aware neural topic model for COVID-19 disinformation categorisation, the architecture generates class-associated word clusters that define the semantic boundaries of categories like disinformation. When the model makes a prediction, users can inspect the specific vocabulary triggering the output.

This design principle is echoed across various text applications:

Vision Modality: Grouping Pixels into Object Parts

The same intuition applies to computer vision. Standard Vision Transformers (ViTs) compute attention between every single pair of image patches, which is both computationally expensive and difficult to interpret. Semantic clustering solves both problems by grouping patches into discrete spatial clusters.

Consider the PaCa-ViT architecture introduced by Ryan Grainger et al. in PaCa-ViT: Learning Patch-to-Cluster Attention in Vision Transformers. Instead of patch-to-patch attention, it implements a "Patch-to-Cluster" mechanism where the model learns to assign image patches to a small set of semantically consistent clusters. Because these assignments are discrete, they can be directly visualized as heatmaps. If the model groups the wheels, windows, and chassis of a car into a single cluster, a researcher can instantly see that the model has formed a high-level conceptual understanding of a "vehicle" without requiring post-hoc explanation tools.

[ Raw Image Patches ] ---> [ Clustering Bottleneck ] ---> [ Discrete Semantic Clusters ]
                                                                   |
                                                                   v
                                                       [ Inherent Heatmap Visualization ]

Taking this a step further, the authors of Image as Set of Points (Xu Ma et al.) abandoned the rigid pixel grid entirely with Context Clusters (CoCs). By treating an image as an unorganized set of points and clustering them based on similarity, the model naturally partitions the image into distinct object parts and background elements. This inherent interpretability is also leveraged in geometric tasks; Khang Truong Giang et al. developed TopicFM: Robust and Interpretable Topic-Assisted Feature Matching, which represents image regions as distributions over latent structural topics (like architectural elements), ensuring that image matching is anchored on semantically consistent regions that humans can easily verify.

Vision-Language Alignment: Mapping Patches to Lexical Tokens

The ultimate realization of semantic clustering occurs at the intersection of vision and language. If visual clusters can be mapped directly to discrete textual tokens, the model achieves a form of "thematic transparency" where an image region is explicitly grounded in human language.

Early work by Zhiwu Lu et al. in Direct Semantic Analysis for Social Image Classification established this by learning a direct correlation matrix between local image patches and user-provided social tags, bypassing uninterpretable latent spaces to enable precise semantic segmentation. In generative contexts, Dan dan Guo et al. coupled visual features with hierarchical, word-based topics in Matching Visual Features to Hierarchical Semantic Topics for Image Paragraph Captioning, ensuring the model's internal themes aligned with the actual visual content.

Modern vision-language pre-training refines this concept by forcing fine-grained alignment during training. The SPARC framework, introduced by Ioana Bica et al. in Improving fine-grained understanding in image-text pre-training, aggregates vision embeddings by grouping image patches based on their semantic similarity to specific words in a caption, creating a localized visual representation for each token.

Similarly, the LexVLA framework by Yifan Li et al. (Unified Lexical Representation for Interpretable Visual-Language Alignment) projects local image patches directly into a language model's token space. Because the patches are mapped to a shared vocabulary of actual words, the system achieves precise spatial interpretability. A researcher does not have to guess what a specific vision embedding represents; they can read the exact ground-truth semantic labels associated with that sub-region of the image.

Why It Works

Semantic clustering works because it replaces continuous, high-dimensional ambiguity with discrete, low-dimensional structure. Continuous vectors can move in infinite directions, making it impossible to assign a single human concept to a single dimension.

By forcing a model to route its representations through a clustering layer, the network must categorize information into a finite set of bins. Whether that structure is a list of top keywords in a text topic model or a spatial heatmap of clustered image patches, it forces the neural network to communicate its internal reasoning using the same discrete building blocks that humans use to understand the world.

Key Foundational Work

SMS Spam Filtering using Probabilistic Topic Modelling and Stacked Denoising Autoencoder (2016) — contributed 1 concept elements to this direction.
Direct Semantic Analysis for Social Image Classification (2014) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_cluster_based_interpretability_cf55e54d,
  title = {cluster-based interpretability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cluster-based-interpretability/rdr_cf55e54d431b4c4a9b1531375efd30a1},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_cluster_based_interpretability_cf55e54d_v1,
  title = {cluster-based interpretability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cluster-based-interpretability/rdr_cf55e54d431b4c4a9b1531375efd30a1/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake