Using Large Language Models and Recruiter Expertise for Optimized Multilingual Job Offer - Applicant CV Matching

Hamit Kavas, Marc Serra-Vidal, Leo WannerIJCAI 2024

Abstract

In a globalized labor market, matching job seekers to relevant opportunities is complicated by linguistic diversity and varying professional terminologies. This paper introduces a "human-in-the-loop" system that combines fine-tuned multilingual classifiers with Large Language Models (LLMs) to optimize the recruitment process. By grounding evaluations in the standardized European Skills, Competences, Qualifications and Occupations (ESCO) taxonomy and incorporating real-time recruiter feedback, the system achieves high matching accuracy across languages while providing transparent, justifiable candidate reports.

Introduction

Recruiters today manage a high volume of resumes and job postings written in multiple languages and formats. While standardized taxonomies like ESCO provide a framework for categorization, they often lack the granularity required to distinguish specific professional nuances—such as the difference between a "data analyst" in finance versus education. Existing automated systems frequently struggle with this lack of detail or fail to incorporate the subjective priorities of human recruiters, leading to qualified candidates being overlooked. This research addresses these challenges by balancing automated precision with human expertise.

Methodology

The proposed system utilizes a multi-stage pipeline designed to harmonize automation with recruiter insight.

  1. Initial Classification: The process begins with ESCOXLM-R+, a state-of-the-art multilingual model fine-tuned to align CVs with ESCO occupation labels. This serves as a high-speed filter to identify relevant candidates.
  2. Skill Extraction: Using Mixtral 8x7B, the system extracts specific skills and competencies from the filtered CVs.
  3. Manual Correction System (MCS): This "human-in-the-loop" component allows recruiters to review extracted skills, mark irrelevant items, or add comments to prioritize specific requirements via an interactive interface.
  4. RAG-Powered Evaluation: To mitigate LLM hallucinations, the system employs Retrieval Augmented Generation (RAG), utilizing a vector store of 3,015 distinct ESCO occupation definitions as a knowledge base.
  5. Scoring and Reporting: The Mixtral LLM evaluates candidates against refined requirements, assigning a score (1-10) and generating a summary report. To ensure stability, scores are averaged over three iterations.
The multi-stage pipeline for matching multilingual CVs to job offers.

Experimental Setup

The system was evaluated using several diverse datasets:

  • InfoJobs Spanish: 28,629 documents from a major Spanish job board.
  • Kaggle English: 27,002 documents for cross-language validation.
  • Training Set: A curated dataset of 63,161 English and 50,521 Spanish documents used for fine-tuning the ESCOXLM-R+ classifier.
  • ESCO Taxonomy: Served as the ground truth for labels and the knowledge base for RAG.

Results

The quantitative performance of the initial classifier demonstrates the value of domain-specific fine-tuning. The fine-tuned ESCOXLM-R+ achieved a Recall@5 of 0.95 for Spanish and 0.98 for English. In contrast, general-purpose models like Multilingual e5-large achieved significantly lower scores, particularly in the Spanish dataset.

Performance comparison (Recall@5) between the proposed ESCOXLM-R+ model and general-purpose embedding baselines.

Qualitatively, the system provides recruiters with intuitive visualizations, such as radar plots, to compare candidate suitability across dimensions like Tech Adaptability, Problem-solving, and Teamwork.

Discussion and Conclusion

The research highlights that specialization in taxonomies like ESCO significantly outperforms general-purpose embeddings. By treating recruiters as "feature engineers" whose feedback reshapes scoring logic, the system acknowledges the subjective nature of recruitment. While the system is robust and handles English, Spanish, and Catalan seamlessly, it faces challenges regarding computational intensity due to the multiple LLM calls required for scoring stability. Future work will focus on exploring more specialized fine-tuned LLMs and "class embeddings" to further refine the categorization of job offers and CVs.

Summary updated

BibTeX
@inproceedings{kavas2024using,
  author = {Hamit Kavas and Marc Serra-Vidal and Leo Wanner},
  title = {{Using Large Language Models and Recruiter Expertise for Optimized Multilingual Job Offer - Applicant CV Matching}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure