Document-based Recommender System for Job Postings using Dense Representations

Ahmed Elsafty, Martin Riedl, Chris BiemannNAACL 2018 · DOI

Introduction

Traditional job recommendation systems often struggle with the limitations of keyword-based matching. These "sparse" representations fail to account for vocabulary mismatches (e.g., "J2EE Developer" vs. "Software Engineer"), rely on overly broad categories, and suffer from significant metadata gaps. To address these issues, researchers at XING and the University of Hamburg developed a "More-Like-This" (MLT) component that utilizes dense vector representations to capture the semantic similarity between job postings.

Proposed Methodology

The core of the system involves a transition from traditional TF-IDF models to dense document embeddings. The researchers evaluated several architectures, including Word2Vec and Doc2VecC. The latter is a robust version of Doc2Vec that uses a corruption-based training objective to improve document-level representations.

Title-Weighted Descriptions

A key innovation of this research is the introduction of a title-weighting scheme. Recognizing that job titles carry the highest semantic signal, the authors calculated document vectors as a weighted average of word vectors. Words were weighted by their TF-IDF scores, with an additional multiplier (constant $c = 5.0$) applied to words appearing in the job title. This approach effectively anchors the document vector to the core intent of the posting, preventing it from being diluted by boilerplate text in the full description.

Experimental Evaluation

To evaluate the system, the authors created a "silver standard" dataset derived from 112,000 distinct job postings and user interactions (bookmarks and reply intentions) on the XING platform. This allowed them to measure how well the models could identify jobs that users found similar in practice.

Offline Performance

Offline tests demonstrated a dramatic improvement over the baseline. While the standard TF-IDF model achieved a Precision@10 of only 8.69%, the introduction of dense representations saw performance climb significantly.

Table 1: Comparison of offline precision (P@10) across different embedding models and weighting schemes.

As shown in the table above, the basic Doc2VecC model reached 64.23%. When the custom title-weighting scheme was applied, the performance further increased to a peak of 73.05% Precision@10.

Results and Discussion

Online Impact and Qualitative Shift

In a live A/B test involving over one million users, the dense representation system delivered an 8.0% relative increase in click-through rate (CTR) compared to the existing Elasticsearch-based ranking. Beyond quantitative metrics, the system exhibited a notable qualitative shift. It moved away from recommending generic roles toward more specific, research-oriented positions that better aligned with user expertise.

Table 2: Qualitative comparison of recommendations between the existing keyword-based system and the new dense representation system.

Implicit Geography and Efficiency

An unexpected benefit of the dense model was its ability to learn implicit geography. Recommendations were found to be 30% to 60% closer to the user's physical location than those of previous systems. This occurred because the model learned the semantic relationships between city names in titles without explicit geographic metadata. Furthermore, the system remains computationally efficient, adding less than 10% to average response times, making it highly suitable for production environments.

Conclusion

By combining dense document embeddings with a novel title-weighting heuristic, the researchers successfully built a recommender system that understands the semantic nuances of job postings. The significant gains in both offline precision and online CTR demonstrate the power of moving beyond keyword matching in the recruitment domain.

Summary updated

BibTeX
@inproceedings{elsafty2018document,
  author = {Ahmed Elsafty and Martin Riedl and Chris Biemann},
  title = {{Document-based Recommender System for Job Postings using Dense Representations}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2018},
  doi = {10.18653/v1/n18-3027},
}
Figure