TIGFORMER at TextGraphs-17 Shared Task: A Late Interaction Method for text and Graph Representations in KBQA Classification Task

Mayank Rakesh, Parikshit Saikia, Saket ShrivastavaACL 2024

TL;DR

This paper introduces TIGFORMER, a novel late interaction method for Knowledge Base Question Answering (KBQA) that effectively fuses text and graph representations. By employing a sophisticated interaction module, TIGFORMER significantly improves performance over traditional baselines, demonstrating the power of integrating diverse data modalities for complex question answering.

What Problem Does This Paper Address?

Answering factoid questions accurately, especially those requiring precise, context-specific information, remains a challenge for even the most advanced language models. While models like BERT excel at processing text, they often struggle without access to structured, contextual data. Knowledge Graphs (KGs) offer a rich source of such data, but effectively integrating them with language models to enhance question answering is an ongoing area of research. Existing methods often limit themselves to using only textual information from KGs or employ simpler integration strategies. This paper tackles the problem of how to best combine the semantic understanding of text with the structured, relational information from knowledge graphs to achieve superior KBQA performance.

Methods

The core of this work is the TIGFORMER framework, which employs a "late interaction" approach. This means that text and graph representations are processed independently first and then brought together for interaction.

Here's a breakdown of the key components:

  • Text Module: A T5-transformer encoder is used to generate text embeddings from the input question.
  • Graph Module: Graphormer, a specialized transformer for graph data, is employed to encode the structural information of sub-graphs. This includes node features, edge encodings, spatial relationships (shortest path distances), and centrality measures.
  • Interaction Module: This is the novel contribution. After pooling text and graph representations (using Mean, CLS, or Attention pooling), a dedicated interaction module facilitates information exchange. It calculates similarity coefficients between text and graph embeddings and uses these weights to create fused representations. The paper specifically highlights an "attention pooling" mechanism as being particularly effective.
  • Classification Module: The final, interacted representations are concatenated and fed into a classification head to determine if the question, answer entity, and sub-graph combination is factually correct.

The paper also compares their proposed interaction function against simpler "Average" and "Soft" functions, finding their method to be superior.

Datasets

The paper utilizes a dataset from the TextGraphs-17 Shared Task (Sakhovskiy et al., 2024). This dataset consists of questions, a list of Wikidata entities mentioned in them, and 5-10 answer candidates (also Wikidata entities). Crucially, it includes a Wikidata sub-graph that connects the entities from the question to the answer candidates via shortest paths.

Key Insights and Ideas

  • Late Interaction is Key: The paper strongly advocates for a "late interaction" approach, where text and graph representations are processed independently before being fused. This allows each modality to be fully understood before integration, leading to richer combined representations.
  • Attention Pooling Shines: Among the pooling strategies explored, attention pooling emerged as the most effective for combining text and graph information. This suggests that dynamically weighting different parts of the representations based on their relevance is crucial.
  • Linearized Graphs and Question Features Matter: The best performance was achieved when using linearized graph and question features as text inputs, combined with sub-graph representations for the graph input. This highlights the importance of how information is presented to the interaction module.
  • Beyond Text-Only or Graph-Only: The results clearly demonstrate that a hybrid approach, effectively integrating both text and graph data, significantly outperforms methods that rely on a single modality.

Key Results

The proposed TIGFORMER method, particularly with attention pooling and linearized graph/question features, achieved a F1-score of 0.5305. This represents a substantial improvement over the baselines:

  • Baseline-graph-only: 0.1266 F1-score
  • Baseline-text-graph: 0.2022 F1-score
  • Baseline-text-only: 0.2120 F1-score

The paper also shows that their proposed interaction function outperforms simpler "Average" and "Soft" interaction functions.

Limitations and Future Work

  • Limitations: The paper doesn't explicitly detail limitations of their proposed method beyond the inherent challenges of KBQA. The performance, while improved, still has room for growth, as indicated by the F1-score.
  • Future Work: The authors emphasize the importance of combining various modalities and using cutting-edge interaction methods to improve KBQA systems. They suggest that further research can enhance model robustness and performance through more efficient integration of textual and graph-based data, paving the way for future advancements in factoid-based question answering.

Summary updated

BibTeX
@inproceedings{rakesh2024tigformer,
  author = {Mayank Rakesh and Parikshit Saikia and Saket Shrivastava},
  title = {{TIGFORMER at TextGraphs-17 Shared Task: A Late Interaction Method for text and Graph Representations in KBQA Classification Task}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure