IERL: Interpretable Ensemble Representation Learning -- Combining CrowdSourced Knowledge and Distributed Semantic Representations

Yuxin Zi, Kaushik Roy, Vignesh Narayanan, Manas Gaur, Amit ShetharXiv 2023 · arXiv

TL;DR

IERL (Interpretable Ensemble Representation Learning) is a novel framework that bridges the gap between the statistical prowess of Large Language Models (LLMs) and the structured reliability of human-curated Knowledge Graphs (KGs). By combining BERT representations with ConceptNet using a unique "moment-based" ensemble approach, the authors achieved state-of-the-art performance on GLUE tasks while making the model’s decision-making process transparent and significantly faster to train.

What Problem Does This Paper Address?

While LLMs like BERT excel at capturing statistical patterns in language, they are prone to hallucinations—generating inconsistent or incorrect outputs when faced with rare inputs or ambiguous contexts. This happens because LLMs rely on "distributed semantics" learned from massive, often messy, datasets.

On the other hand, Knowledge Graphs (KGs) like ConceptNet offer stable, human-agreed-upon definitions of words. The challenge has been finding a way to "infuse" this structured knowledge into LLMs without creating a "black box" where it's impossible to tell why the model made a specific choice. Existing methods often modify the model's internal parameters in ways that are hard to audit, leaving researchers in the dark about whether the model is relying on factual knowledge or just statistical noise.

Methods

The authors introduce IERL, which treats the combination of LLM and KG data as an ensemble learning problem at the instance level.

Key innovations include:

  1. Dual Representations: For every input sentence, the model generates two sets of embeddings: one from an LLM (BERT) and one from a KG (ConceptNet).
  2. Moment-Based Aggregation: Instead of simple averaging, IERL uses higher-order moments (up to the 4th order, including variance and skewness) to aggregate representations. The authors argue that these higher-order moments act as "sufficient statistics" that better describe the data distribution, especially in smaller datasets like those in the GLUE benchmark.
  3. Interpretable Optimization: The model learns a set of parameters ($\alpha$) that weight the importance of the LLM vs. the KG for each specific input. Because these weights are explicit, researchers can see exactly which source the model prioritized for a given task.
The IERL workflow, showing how BERT and ConceptNet representations are aggregated and optimized.

Datasets

The model was evaluated using the GLUE (General Language Understanding Evaluation) benchmark, a standard suite for testing natural language comprehension. Specifically, they tested on:

  • Sentence Similarity: QQP (Quora Question Pairs) and STS (Semantic Textual Similarity).
  • Sentence Entailment: QNLI, WNLI, MNLI, and RTE.

Key Insights and Ideas

  • Higher-Order Moments are a Game Changer: One of the most surprising findings was that using higher-order moments (Algorithm 2) dramatically accelerated convergence. The model reached optimal performance in just 7-13 steps, compared to 20-45 steps for vanilla BERT.
  • Interpretability by Design: Unlike "post-hoc" interpretation methods (like LIME) that try to guess why a model worked after the fact, IERL is interpretable by construction. The $\alpha$ parameters provide a direct window into the model's "thought process."
  • Stability via Grounding: By grounding the LLM's fluid statistical patterns in the KG's rigid factual structure, the model becomes more robust against the irregularities that typically cause hallucinations.
Comparison of simple averaging (baseline) versus the proposed moment-based aggregation.

Key Results

IERL demonstrated competitive or superior performance across the board:

  • Accuracy: It achieved a remarkable 98.7% on WNLI (compared to BERT's 93.3%) and 93.55% on STS.
  • Efficiency: As shown in the table below, IERL stabilized in significantly fewer optimization steps (#Opt) than vanilla BERT or the simpler baseline version ($IERL_B$).
  • Qualitative Clarity: The authors provided visual examples (Figure 2) showing how the model distinguishes between similar historical facts, highlighting when it leans on BERT's context versus ConceptNet's definitions.
Quantitative results showing IERL's performance and convergence speed across GLUE tasks.

Limitations and Future Work

While the results are promising, the authors acknowledge a few areas for growth:

  • Choice of Models: The current study is limited to BERT and ConceptNet. Future work will explore how IERL performs with other LLMs (like GPT variants) and different types of KGs.
  • Combination Functions: The current ensemble uses a linear combination. The team plans to investigate more complex, non-linear functions that could capture hierarchical relationships within KGs more effectively.
  • Moment Selection: Further research is needed to determine the optimal "order" of moments for different types of linguistic tasks.

Summary updated

Abstract

Large Language Models (LLMs) encode meanings of words in the form of distributed semantics. Distributed semantics capture common statistical patterns among language tokens (words, phrases, and sentences) from large amounts of data. LLMs perform exceedingly well across General Language Understanding Evaluation (GLUE) tasks designed to test a model's understanding of the meanings of the input tokens. However, recent studies have shown that LLMs tend to generate unintended, inconsistent, or wrong texts as outputs when processing inputs that were seen rarely during training, or inputs that are associated with diverse contexts (e.g., well-known hallucination phenomenon in language generation tasks). Crowdsourced and expert-curated knowledge graphs such as ConceptNet are designed to capture the meaning of words from a compact set of well-defined contexts. Thus LLMs may benefit from leveraging such knowledge contexts to reduce inconsistencies in outputs. We propose a novel ensemble learning method, Interpretable Ensemble Representation Learning (IERL), that systematically combines LLM and crowdsourced knowledge representations of input tokens. IERL has the distinct advantage of being interpretable by design (when was the LLM context used vs. when was the knowledge context used?) over state-of-the-art (SOTA) methods, allowing scrutiny of the inputs in conjunction with the parameters of the model, facilitating the analysis of models' inconsistent or irrelevant outputs. Although IERL is agnostic to the choice of LLM and crowdsourced knowledge, we demonstrate our approach using BERT and ConceptNet. We report improved or competitive results with IERL across GLUE tasks over current SOTA methods and significantly enhanced model interpretability.
BibTeX
@misc{zi2023ierl,
  author = {Yuxin Zi and Kaushik Roy and Vignesh Narayanan and Manas Gaur and Amit Sheth},
  title = {{IERL: Interpretable Ensemble Representation Learning -- Combining CrowdSourced Knowledge and Distributed Semantic Representations}},
  year = {2023},
  eprint = {2306.13865},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2306.13865},
}
Figure