Grouping Entities with Shared Properties using Multi-Facet Prompting and Property Embeddings

Amit Gajbhiye, Thomas Bailleux, Zied Bouraoui, Luis Espinosa-Anke, Steven SchockaertEMNLP 2025

TL;DR

This paper introduces Multi-Facet Prompting (MFP), a novel method for discovering commonalities among large sets of entities by leveraging Large Language Models (LLMs) to generate diverse properties. By clustering these properties, MFP effectively groups entities and demonstrates significant improvements in downstream tasks like ontology completion and ultra-fine entity typing.

What Problem Does This Paper Address?

The paper tackles the challenge of "commonality identification": given a set of entities, how can we automatically discover meaningful ways to group them based on shared properties? While LLMs intuitively seem capable of this, directly applying them to large domains (tens of thousands of entities) is computationally infeasible. Existing methods, like those using BERT bi-encoders, struggle with the diversity of commonsense properties needed for robust grouping. The core problem is to efficiently and effectively extract nuanced, shared characteristics from entities in a scalable manner.

Methods

The proposed approach, Multi-Facet Prompting (MFP), uses LLMs indirectly. First, an LLM is prompted to generate properties for each entity, crucially guided by different "facets" of knowledge (e.g., for "banana," facets could include "category," "color," "shape," "nutritional content"). This multi-facet prompting encourages a richer and more diverse set of properties compared to single-facet approaches.

These generated properties are then verbalized into text strings (e.g., "category: fruit"). These strings are encoded into embeddings using a pre-trained LLM2Vec model. Finally, affinity propagation is used to cluster these property embeddings. Each cluster of properties defines an entity category, comprising all entities for which at least one property from that cluster was predicted. The authors explore various verbalization strategies and clustering methods, finding that facet-based prompting and affinity propagation yield the best results.

Datasets

The authors evaluate their method on several datasets:

  • Commonsense (CS) dataset: Adapted from an outlier detection dataset, focusing on everyday concepts and commonsense properties.
  • SemEval Taxonomies: Datasets from TexEval-1 and TexEval-2 (Food, Science, Equipment, Environment) using leaf nodes from gold taxonomies.
  • Wikidata Benchmark: A newly constructed dataset comprising 42 problem instances across six entity types (humans, politicians, cathedrals, mountains, songs, games), each defined by an entity type, a Wikidata property, and a value.

Key Insights and Ideas

  • Facets are Crucial: The paper strongly emphasizes that prompting LLMs to generate properties across different facets of knowledge is key to uncovering diverse and meaningful commonalities. Without facets, the generated properties are less diverse and the grouping is less effective.
  • LLMs as Property Generators, Not Direct Groupers: The approach cleverly sidesteps the scalability issues of directly using LLMs for grouping by employing them as sophisticated property extractors, whose outputs are then processed by more scalable clustering methods.
  • Clustering Embeddings Captures Nuance: By clustering embeddings of properties (rather than entities directly), the method can identify shared characteristics that might not be immediately obvious from entity embeddings alone.
  • Literal Matches are Insufficient: A variant of MFP that omits the clustering step (literal match) performs poorly, highlighting the necessity of the clustering mechanism to consolidate similar properties.

Key Results

  • Outperforms Baselines: MFP consistently outperforms several baselines, including LLM2Vec clusters, MClu (multi-facet concept embeddings), and TaxoLlama (a hypernym discovery model), across intrinsic evaluation tasks.
  • Ontology Completion: MFP achieves substantial and consistent gains on the ontology completion task, outperforming state-of-the-art methods.
  • Ultra-fine Entity Typing (UFET): The method also shows significant improvements on UFET, demonstrating its ability to identify relevant categories that can act as semantic regularizers for training.
  • Facet Importance: The "MFP (no facets)" variant performs significantly worse than the full MFP model, underscoring the importance of facet-based prompting.
  • Prompt Iterations Matter: Repeating the LLM prompt multiple times (e.g., 10 iterations) significantly boosts performance, suggesting that a more diverse set of generated properties leads to better results.

Limitations and Future Work

  • LLM Choice and Prompting: The paper acknowledges that the performance might be further improved by experimenting with different LLMs and more sophisticated prompting strategies.
  • Constructing Full Taxonomies: While the method identifies categories, it doesn't construct full hierarchical taxonomies (trees or DAGs). The authors suggest that hierarchical clustering might be a direction for future work.
  • Ambiguity and Incoherence: Qualitative analysis reveals issues with ambiguous entity names leading to irrelevant properties, and sometimes incoherent clusters due to property clustering.
  • Computational Overhead: Fine-tuning LLMs or using iterative approaches to refine commonalities could increase computational cost.
  • Post-processing: The authors suggest that post-processing entity clusters with Natural Language Inference (NLI) models could help filter spurious entities, similar to prior work.

Summary updated

BibTeX
@inproceedings{gajbhiye2025grouping,
  author = {Amit Gajbhiye and Thomas Bailleux and Zied Bouraoui and Luis Espinosa-Anke and Steven Schockaert},
  title = {{Grouping Entities with Shared Properties using Multi-Facet Prompting and Property Embeddings}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}

Loading related content...

Figure