Beyond Job Titles: Interpreting Early-Career Job Advertisements with Sparse Autoencoders and Semantic Clustering

Yisong Cao, Leah Kim, Brian Wright, Ali RiveraarXiv 2026 · arXiv

TL;DR

This paper introduces a framework to look beyond generic job titles by using semantic clustering and Sparse Autoencoders (SAEs) to analyze over 36,000 job postings. By segmenting advertisements into team context, specific tasks, and required qualifications, the authors reveal the distinct "technical DNA" of roles like Data Scientist and ML Engineer that simple keyword counting often misses.

What Problem Does This Paper Address?

Job titles are notoriously "leaky." A "Data Scientist" at a startup might spend their day building data pipelines, while the same title at a legacy bank might involve mostly Excel-based reporting. Traditional quantitative methods, like counting how often "Python" or "SQL" appears, fail to capture the context of how those tools are used.

The authors aim to move "Beyond Job Titles" to understand how technical roles actually differ in their day-to-day responsibilities and industry applications. They specifically target the early-career market (internships and entry-level roles) to see how the advent of AI is reshaping the expectations for the next generation of technical talent.

Methods

The researchers designed and compared two primary deep-learning-based workflows:

  1. PERC (Preprocessing, Embedding, Reduction, and Clustering):
    • Preprocessing: They used GPT-6 Luna to segment raw job descriptions into three distinct categories: Team context, Role task, and Qualifications.
  2. Embedding & Reduction: Passages were converted into vectors using models like MPNet and text-embedding-3-small, then reduced in dimensionality via PCA or UMAP.
  3. Clustering: They applied various algorithms (K-means, HDBSCAN, etc.) to find semantic groupings.
  4. Sparse Autoencoders (SAEs):
    • The team used a pretrained 537-million-parameter SAE from Goodfire to extract interpretable features from the internal representations (layer 19) of Llama-3.1-8B-Instruct as it "read" the job ads.
  5. This allowed them to identify specific "features" (e.g., "developing machine learning models" or "data reporting") that activate across different descriptions.

To validate their findings, they used Maximum Mean Discrepancy (MMD) to rank the most significant distributional differences between roles and employed GPT-6.1 Terra to verify if these automated insights held true in a held-out set of advertisements from companies like Amazon, Google, and Microsoft.

Figure 1: A PERC workflow diagram that follows job advertisements from preparation to cluster comparison and language-model interpretation. The upper row demonstrates our design for how a typical job receives preprocessing, section segmentation, embedding, dimensionality reduction and clustering. The lower row compares two cluster distributions and ranks their MMD scores before GPT-6.1 Terra reads the selected advertisements. The narrative example shown in the figure contrasts December 2025 descriptions of more “supervised work” with August 2026 descriptions of “greater operational responsibility” generated via an LLM verifier at the end.

Datasets

  • Primary Dataset: 36,081 early-career job postings scraped from LinkedIn between December 2025 and October 2026.
  • Composition: The sample is heavily skewed toward internships (35,511) compared to other early-career roles (570).
  • Occupations: The analysis focused on five key roles: Data Scientist (DS), Data Analyst (DA), Software Engineer (SWE), Machine Learning Engineer (MLE), and Business Analyst (BA).

Key Insights and Ideas

  • Keywords vs. Context: Everyone asks for Python, but the intent differs. In DS roles, Python is for "developing machine learning models"; in SWE roles, it's for "software development"; and in DA roles, it's often for "data reporting."
  • The MLE vs. DS Distinction: SAE features revealed that Machine Learning Engineer roles place a much higher emphasis on the lifecycle and productionization of models (e.g., "AI-system implementation"), whereas Data Scientist roles are more focused on using models to answer analytical questions.
  • Segmentation is Crucial: The authors found that clustering "Qualifications" separately from "Role Tasks" revealed a gap: companies often request high-level degrees (Qualifications) for roles where the actual work (Tasks) is more operational or reporting-heavy.
  • Interpretable Latent Spaces: Using SAEs proved to be a powerful way to "peek" into an LLM's brain. Instead of just seeing that two ads are "similar" in vector space, the SAE identifies why—for instance, because both mention "LLM and agent workflow development."
Figure 5: A series of PCA scatter plots of PERC clusters for Team context, Role task and Qualifications. Each point represents one job advertisement and each color represents a cluster, with gray representing unassigned advertisements. We can see the clearest colored regions in Qualifications, with the blue cluster concentrated on the right, the green cluster at the lower left and the pink and gold clusters above. Role task also has broad colored regions but more overlap around their boundaries, whereas Team context has several colors mixed through its center. The first two PCA components explain 11.6%–14.8% of the variance in the original job-ad vectors.

Key Results

  • Clustering Performance: The combination of "Role task + Qualifications" yielded the highest word separation (TF-IDF cosine silhouette of 0.064), suggesting these two sections define a role's identity most strongly.
  • Feature Activation: The SAE feature for "Model development" activated in 79.2% of Data Scientist ads and 71.0% of ML Engineer ads, but only 29.1% of Data Analyst ads.
  • Reporting vs. Analysis: "Analysis and reporting" features were dominant in Business Analyst (85.0%) and Data Analyst (70.7%) roles.
  • Semantic Overlap: Figure 3 shows that Data Analysts and ML Engineers share a relatively high cosine similarity (0.65) in their tasks, likely due to shared data pipeline responsibilities, while Business Analysts and Software Engineers have the lowest overlap (0.38).
Figure 3: A cosine-similarity matrix comparing five Role task advertisements embedded with text-embedding-3-small. The adjacent excerpts show the tasks described in those advertisements. Among these five Role task examples, Data Analyst and Machine Learning Engineer have the highest cosine similarity at 0.65 and Business Analyst and Software Engineer have the lowest at 0.38. The excerpts contrast data pipelines and application development with marketing analysis and reporting, showing how we can compare the work described beneath different occupation names.

Limitations and Future Work

  • Internship Bias: Because the dataset is ~98% internships, the findings might not perfectly generalize to mid-career or senior roles.
  • Data Gaps: Technical outages led to missing data between May and August 2026, which may have obscured some seasonal hiring trends.
  • LLM Dependency: The methodology relies heavily on the accuracy of GPT-6 for segmentation and verification. While the authors used a "held-out company" strategy to mitigate bias, the "LLM-judging-LLM" loop remains a point of scrutiny.
  • Future Directions: The authors suggest expanding the dataset to include more full-time roles and tracking the same companies over multiple years to observe how specific roles (like "AI Engineer") emerge and stabilize.

Summary updated

Abstract

Job titles and job descriptions alone provide limited insight into how technical roles differ in a changing job market. To assess how such roles differ in their work and industry applications, we design and compare two methods based on deep learning and transformer representations using 36,081 early-career job postings collected from LinkedIn in the United States between December 2025 and October 2026. We first apply PERC, short for Preprocessing (P), Embedding (E), Dimensionality Reduction (R) and Clustering (C). We employ GPT-6 Luna to classify job description passages into Team context, Role task and Qualifications. We then create combinations of these sections and examine whether embedding and clustering the segmented passages produce meaningful semantic clusters about different job roles. We use Maximum Mean Discrepancy (MMD) to rank observed differences between embedding distributions and employ a pretrained sparse autoencoder (SAE) to extract features from layer-19 representations of Llama-3.1-8B-Instruct. Lastly, we employ GPT-6 Luna to come up with hypotheses about job roles' feature patterns, and GPT-6.1 Terra to investigate whether those patterns exist in held-out job descriptions from unseen companies. Some example findings include Data Scientist advertisements placing more emphasis on machine learning model development, Data Analyst advertisements on reporting business performance, and Machine Learning Engineer advertisements on applying developed models in real-world settings. We also observe differences in the applications described by Data Analyst and Business Analyst advertisements across selected months, although our dataset's scale and uneven coverage limit their interpretation. Overall, we present a realistic application of SAEs and PERC to unstructured textual data and examine how automated insight extraction could support manual analysis of job descriptions.
BibTeX
@misc{cao2026beyond,
  author = {Yisong Cao and Leah Kim and Brian Wright and Ali Rivera},
  title = {{Beyond Job Titles: Interpreting Early-Career Job Advertisements with Sparse Autoencoders and Semantic Clustering}},
  year = {2026},
  eprint = {2611.21586},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21586},
}
Figure