Building Data-Driven Occupation Taxonomies: A Bottom-Up Multi-Stage Approach via Semantic Clustering and Multi-Agent Collaboration

Nan Li, Bo Ram Kang, Tijl De Bie2025 · DOI

TL;DR

This paper introduces CLIMB, a novel framework that automatically builds detailed occupation taxonomies from raw job postings. By combining global semantic clustering with a multi-agent reflection system, CLIMB creates more coherent, scalable, and regionally specific taxonomies than existing methods, addressing the limitations of slow manual curation and rigid automated approaches.

What Problem Does This Paper Address?

Creating accurate and up-to-date occupation taxonomies is crucial for many applications, like job recommendations and labor market analysis. However, existing methods fall short: manual curation is slow and expensive, while automated approaches are either "top-down" (requiring a pre-existing structure and thus prone to bias) or "bottom-up" (struggling to build coherent hierarchies from noisy, real-world data). Specifically, bottom-up methods often fail to distill a globally consistent set of core concepts and struggle with the complex reasoning needed to build a logical hierarchy. This paper aims to bridge this gap by fully automating the creation of high-quality, data-driven, and adaptable occupation taxonomies directly from raw job postings.

Methods

CLIMB employs a two-stage, bottom-up approach:

  1. Distillation and Semantic Clustering (Leaf Node Generation):
    • Job Posting Distillation: An optional step uses a large language model (LLM) to create training data for a lightweight classifier. This classifier then filters out irrelevant text from job postings, focusing on core occupational details.
  2. Semantic Clustering: Instead of relying on generic embeddings, CLIMB trains a custom similarity metric using an LLM acting as an HR expert to label job pairs. This learned metric is then used with Affinity Propagation to group job postings into fine-grained occupation clusters (potential leaf nodes).
  3. Leaf Node Generation: An LLM abstracts these clusters into canonical occupation titles and descriptions. A normalization and deduplication process further refines these leaf nodes to ensure they are distinct and well-defined.
  4. Hierarchical Taxonomy Construction (Multi-Agent Collaboration):
    • Reflection-Based Multi-Agent System: CLIMB uses a "Generator" LLM to propose parent concepts for each level of the hierarchy and an "Evaluator" agent (rule-based) to critically check for logical consistency (e.g., completeness, exclusivity, validity).
  5. Iterative Refinement: This "Generate-Evaluate Cycle" repeats until the hierarchy at each level is validated, ensuring coherence and preventing error propagation. The process builds the taxonomy level by level, from specific leaf nodes to broader categories.

Datasets

The authors used three real-world datasets for training and evaluation:

  • Palestine Dataset: 2,701 job postings (English and Arabic).
  • Botswana Dataset: 4,854 job postings.
  • USA Dataset: 11,285 job postings.

These datasets were collected from major job search websites and were split into training (90%) and testing (10%) sets. No new benchmarks were introduced, but the authors compare CLIMB against existing methods on these diverse datasets.

Key Insights and Ideas

  • Custom Similarity for Occupations: Generic cosine similarity isn't enough for nuanced occupation grouping. Training a model to mimic HR expert judgment on job pairs is key to capturing subtle similarities.
  • LLM as HR Expert: Leveraging LLMs as proxies for human experts (for both data annotation and taxonomy generation) is a scalable and effective strategy for complex tasks like this.
  • Reflection for Hierarchy: Building a coherent hierarchy is a complex reasoning task. A multi-agent system with a "Generator" and "Evaluator" that iteratively refines the structure is superior to single-pass LLM approaches.
  • Data-Driven Adaptability: CLIMB's bottom-up, data-driven nature allows it to capture unique regional characteristics and emerging trends, making it more adaptive than static, expert-curated taxonomies.

Key Results

  • Coherence and Scalability: CLIMB produces taxonomies that are significantly more coherent and scalable than baselines like TnT and TnT-H, as measured by inter-annotator agreement (Fleiss' kappa). For example, on the Palestine dataset, CLIMB achieved substantial agreement (0.73*) at the leaf level, outperforming TnT-H (0.60).
  • Comprehensiveness and Efficiency: CLIMB strikes a superior balance between covering jobs (Coverage Rate) and utilizing its labels (Label Utilization Rate), outperforming ESCO (high coverage, low utilization) and TnT (high utilization, lower coverage). On the USA dataset, CLIMB achieved 97.34% coverage and 82.28% utilization at L2.
  • Regional Specificity: Qualitative analysis showed CLIMB capturing unique regional roles (e.g., humanitarian roles in Palestine, diamond industry roles in Botswana, and specific education structures in the USA), which are missed by generic taxonomies like ESCO.
  • Hierarchical Quality: CLIMB's multi-agent framework led to superior hierarchical agreement across all levels compared to TnT-H, demonstrating the effectiveness of its iterative refinement process.

Limitations and Future Work

  • LLM Proxy Reliance: The study relies on LLMs as proxies for HR experts and annotators. While a diverse panel was used, potential biases or errors in LLM judgments could still influence results. Future work could involve large-scale validation with human domain experts.
  • Clustering Scalability: Affinity Propagation has quadratic time complexity, which might challenge scalability on extremely large corpora. Future work could explore alternatives like HDBSCAN.
  • Computational Cost: Creating the hierarchical variant of TnT (TnT-H) was computationally expensive, limiting its application to all datasets.
  • LLM Capabilities: The performance of CLIMB is inherently tied to the capabilities of the LLMs used. Advancements in LLM reasoning could further improve taxonomy quality.
  • Domain Agnosticism: While CLIMB was developed for job postings, its core principles (distilling concepts from noisy data and organizing them with a reasoning engine) could be adapted to other domains like scientific literature or product catalogs.

Summary updated

Abstract

Creating robust occupation taxonomies, vital for applications ranging from job recommendation to labor market intelligence, is challenging.Manual curation is slow, while existing automated methods are either not adaptive to dynamic regional markets (top-down) or struggle to build coherent hierarchies from noisy data (bottom-up). We introduce CLIMB (CLusterIng-based Multi-agent taxonomy Builder), a framework that fully automates the creation of high-quality, data-driven taxonomies from raw job postings. CLIMB uses global semantic clustering to distill core occupations, then employs a reflection-based multi-agent system to iteratively build a coherent hierarchy. On three diverse, real-world datasets, we show that CLIMB produces taxonomies that are more coherent and scalable than existing methods and successfully capture unique regional characteristics. We release our code and datasets at https://github.com/aida-ugent/CLIMB.
BibTeX
@misc{li2025building,
  author = {Nan Li and Bo Ram Kang and Tijl De Bie},
  title = {{Building Data-Driven Occupation Taxonomies: A Bottom-Up Multi-Stage Approach via Semantic Clustering and Multi-Agent Collaboration}},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-industry.113},
}
Figure