Taxonomy Builder: a Data-driven and User-centric Tool for Streamlining Taxonomy Construction

Mihai Surdeanu, John L. Hungerford, Yee Seng Chan, Jessica MacBride, Benjamin M. Gyori, Andrew Zupon, Zheng Tang, Haoling Qiu, Bonan Min, Yan Zverev, Caitlin Hilverman, Max Thomas, Walter S. Andrews, Keith Alcock, Zeyu Zhang, Michael Reynolds, Steven Bethard, Rebecca Sharp, Egoitz Laparra2022 · DOI

Abstract

Mihai Surdeanu, John Hungerford, Yee Seng Chan, Jessica MacBride, Benjamin Gyori, Andrew Zupon, Zheng Tang, Haoling Qiu, Bonan Min, Yan Zverev, Caitlin Hilverman, Max Thomas, Walter Andrews, Keith Alcock, Zeyu Zhang, Michael Reynolds, Steven Bethard, Rebecca Sharp, Egoitz Laparra. Proceedings of the Second Workshop on Bridging Human--Computer Interaction and Natural Language Processing.

2022.

Introduction

In the field of Information Extraction (IE), taxonomies serve as the essential knowledge backbones that categorize and ground extracted data. However, building these hierarchies is traditionally a manual, labor-intensive process that struggles to keep pace with the volume of new data. Taxonomy Builder addresses this bottleneck by providing a data-driven, human-in-the-loop tool designed to accelerate the creation and extension of domain-specific taxonomies.

System Architecture and Methods

The tool utilizes a hybrid architecture that balances heavy computational tasks with a responsive user interface, divided into offline and online phases.

Offline Pre-processing

To filter noise and identify relevant concepts, the system first performs extractive summarization on the corpus using a Luhn-inspired technique. Noun and verb phrases are then extracted from these salient sentences and ranked via an extended TextRank algorithm. Finally, the top-ranked phrases are encoded using DistilBERT contextualized embeddings and grouped using the Clustering By Committee (CBC) algorithm. This ensures that suggested concepts are semantically coherent and relevant to the specific domain.

System architecture of Taxonomy Builder, illustrating the offline pre-processing and online interactive components.

Online Interactive Session

The web-based UI allows users to curate the generated suggestions. The workflow is split between two main components:

  1. Cluster Viewer: Users review suggested clusters, selecting high-quality phrases and rejecting noise.
  2. Taxonomy Editor: Users place the accepted concepts into the existing hierarchical structure.

The Curation Workflow

The core of the user experience is the Cluster Viewer, which presents ranked concepts for human evaluation. By keeping the human "in the loop," the system ensures high precision while significantly reducing the manual effort required to discover new nodes in a large corpus.

The Cluster Viewer interface, where users evaluate and curate suggested concept clusters.

Evaluation and Results

The tool was tested on real-world scenarios involving food insecurity and regional security. Key findings include:

  • Efficiency: Experts were able to process an average of 200 taxonomy candidates per hour.
  • Quality: Approximately 16% of suggested clusters were deemed high-quality enough to be added to formal taxonomies.
  • Improved Grounding: In food insecurity studies, the updated taxonomy led to a 7.6% relative increase in the correlation between grounding confidence and correctness.
  • Downstream Impact: For regional security, the tool enabled the discovery of 33.8% more nodes and 27.6% more causal edges in knowledge graphs.

Conclusion

Taxonomy Builder demonstrates that combining automated summarization and clustering with human curation can significantly streamline taxonomy construction. By leveraging contextualized embeddings and a user-centric design, the tool provides a scalable solution for maintaining the knowledge structures necessary for modern IE systems.

Summary updated

BibTeX
@misc{surdeanu2022taxonomy,
  author = {Mihai Surdeanu and John L. Hungerford and Yee Seng Chan and Jessica MacBride and Benjamin M. Gyori and Andrew Zupon and Zheng Tang and Haoling Qiu and Bonan Min and Yan Zverev and Caitlin Hilverman and Max Thomas and Walter S. Andrews and Keith Alcock and Zeyu Zhang and Michael Reynolds and Steven Bethard and Rebecca Sharp and Egoitz Laparra},
  title = {{Taxonomy Builder: a Data-driven and User-centric Tool for Streamlining Taxonomy Construction}},
  year = {2022},
  doi = {10.18653/v1/2022.hcinlp-1.1},
}
Figure