NEO: A System for Identifying New Emerging Occupation from Job Ads

Anna Giabelli, Lorenzo Malandri, Fabio Mercorio, Mario Mezzanzanica, Andrea SevesoAAAI 2021 · DOI

Introduction

Standard labor market taxonomies, such as the European Skills, Competences, Qualifications and Occupations (ESCO), serve as the essential "lingua franca" for economists and policymakers to track employment trends. However, these taxonomies are typically maintained by expert panels through manual processes that are slow, expensive, and often lag behind the rapid evolution of the digital economy. When roles like "Data Scientist" or "Cloud Architect" emerge, it can take years for them to be formally recognized and integrated into official frameworks.

NEO (New Emerging Occupations) is an automated system designed to bridge this gap. By mining millions of online job advertisements and combining deep learning-based word embeddings with a novel hierarchical similarity metric, NEO identifies emerging job titles and suggests exactly where they should fit within existing professional hierarchies.

System Architecture

The NEO system follows a structured three-step workflow to move from raw web data to an enriched taxonomy.

The three-step NEO workflow: Learning embeddings, suggesting entities, and human-in-the-loop enrichment.

1. Learning Word Embeddings

The system trains three different models—Word2Vec, GloVe, and FastText—on a massive corpus of job advertisements. To improve upon standard similarity measures, the authors introduced Hierarchical Semantic Relatedness (HSR). Unlike traditional metrics that treat all words equally, HSR accounts for the taxonomy's structure and the "density" of concepts (the number of child entities a node has). This ensures that generic terms are not incorrectly equated with highly specific ones.

2. Suggesting New Entities

NEO extracts potential new titles and ranks them using the GASC measures, a suite of four metrics designed to evaluate the fit of a new term:

  • Generality: Measures how similar a term is to unrelated concepts.
  • Adequacy: Determines the overall "fitness" of the term for a specific category.
  • Specificity: Evaluates how closely the term relates to its immediate parent concept.
  • Comparability: Assesses how many skills the new term shares with existing occupations in that category.

3. Confirm and Enrich

A "human-in-the-loop" interface allows experts to review these suggestions. The system provides a visualization of "skill gaps" using Revealed Comparative Advantage (RCA) to show exactly which skills the new occupation shares with its proposed parent category and where it differs.

The NEO evaluation interface showing GASC metrics (Adequacy, Specificity, Generality, Comparability) for suggested occupations.

Datasets and Evaluation

The system was evaluated using a subset of data from the CEDEFOP (2016) project, consisting of over 2.1 million online job advertisements published in the United Kingdom in 2018. The baseline taxonomy used for enrichment was ESCO.

Key findings from the evaluation include:

  • Expert Validation: In a study with 10 experts, 88% of the mentions (43 out of 49) identified by NEO were successfully validated as genuine new occupations.
  • Statistical Reliability: Researchers found a strong positive correlation (using Spearman’s $\rho$ and Kendall’s $\tau$) between the automated GASC scores and the subjective ratings provided by human experts, proving the metrics are reliable proxies for human judgment.
  • Model Performance: The fastText architecture using the Continuous Bag-of-Words (CBOW) algorithm was identified as the most effective model for this specific labor-market domain.

Key Insights

NEO demonstrates that context is king in labor market analysis. By leveraging "distributional semantics"—the idea that words appearing in similar contexts have similar meanings—the system bridges the gap between informal job ad language and formal taxonomic definitions. Furthermore, by acknowledging that hierarchy matters, NEO’s HSR metric represents a significant step forward in NLP for taxonomies, recognizing that broad roles (e.g., "Software Developer") and specific roles (e.g., "React Native Engineer") require different handling.

Limitations and Future Work

While NEO is designed to be language-agnostic, the current results are limited to English-language UK job ads. Additionally, the system still requires a final human "vote" to maintain the integrity of de-jure taxonomies. Future work aims to extend the system to handle the 32 languages covered by the CEDEFOP project and to track how emerging occupations evolve into standard roles over several years of longitudinal data.

Summary updated

BibTeX
@inproceedings{giabelli2021neo,
  author = {Anna Giabelli and Lorenzo Malandri and Fabio Mercorio and Mario Mezzanzanica and Andrea Seveso},
  title = {{NEO: A System for Identifying New Emerging Occupation from Job Ads}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i18.18004},
}
Figure