TL;DR
This paper introduces a framework to look beyond generic job titles by using semantic clustering and Sparse Autoencoders (SAEs) to analyze over 36,000 job postings. By segmenting advertisements into team context, specific tasks, and required qualifications, the authors reveal the distinct "technical DNA" of roles like Data Scientist and ML Engineer that simple keyword counting often misses.
What Problem Does This Paper Address?
Job titles are notoriously "leaky." A "Data Scientist" at a startup might spend their day building data pipelines, while the same title at a legacy bank might involve mostly Excel-based reporting. Traditional quantitative methods, like counting how often "Python" or "SQL" appears, fail to capture the context of how those tools are used.
The authors aim to move "Beyond Job Titles" to understand how technical roles actually differ in their day-to-day responsibilities and industry applications. They specifically target the early-career market (internships and entry-level roles) to see how the advent of AI is reshaping the expectations for the next generation of technical talent.
Methods
The researchers designed and compared two primary deep-learning-based workflows:
- PERC (Preprocessing, Embedding, Reduction, and Clustering):
- Preprocessing: They used GPT-6 Luna to segment raw job descriptions into three distinct categories: Team context, Role task, and Qualifications.
- Embedding & Reduction: Passages were converted into vectors using models like MPNet and text-embedding-3-small, then reduced in dimensionality via PCA or UMAP.
- Clustering: They applied various algorithms (K-means, HDBSCAN, etc.) to find semantic groupings.
- Sparse Autoencoders (SAEs):
- The team used a pretrained 537-million-parameter SAE from Goodfire to extract interpretable features from the internal representations (layer 19) of Llama-3.1-8B-Instruct as it "read" the job ads.
- This allowed them to identify specific "features" (e.g., "developing machine learning models" or "data reporting") that activate across different descriptions.
To validate their findings, they used Maximum Mean Discrepancy (MMD) to rank the most significant distributional differences between roles and employed GPT-6.1 Terra to verify if these automated insights held true in a held-out set of advertisements from companies like Amazon, Google, and Microsoft.
Datasets
- Primary Dataset: 36,081 early-career job postings scraped from LinkedIn between December 2025 and October 2026.
- Composition: The sample is heavily skewed toward internships (35,511) compared to other early-career roles (570).
- Occupations: The analysis focused on five key roles: Data Scientist (DS), Data Analyst (DA), Software Engineer (SWE), Machine Learning Engineer (MLE), and Business Analyst (BA).
Key Insights and Ideas
- Keywords vs. Context: Everyone asks for Python, but the intent differs. In DS roles, Python is for "developing machine learning models"; in SWE roles, it's for "software development"; and in DA roles, it's often for "data reporting."
- The MLE vs. DS Distinction: SAE features revealed that Machine Learning Engineer roles place a much higher emphasis on the lifecycle and productionization of models (e.g., "AI-system implementation"), whereas Data Scientist roles are more focused on using models to answer analytical questions.
- Segmentation is Crucial: The authors found that clustering "Qualifications" separately from "Role Tasks" revealed a gap: companies often request high-level degrees (Qualifications) for roles where the actual work (Tasks) is more operational or reporting-heavy.
- Interpretable Latent Spaces: Using SAEs proved to be a powerful way to "peek" into an LLM's brain. Instead of just seeing that two ads are "similar" in vector space, the SAE identifies why—for instance, because both mention "LLM and agent workflow development."
Key Results
- Clustering Performance: The combination of "Role task + Qualifications" yielded the highest word separation (TF-IDF cosine silhouette of 0.064), suggesting these two sections define a role's identity most strongly.
- Feature Activation: The SAE feature for "Model development" activated in 79.2% of Data Scientist ads and 71.0% of ML Engineer ads, but only 29.1% of Data Analyst ads.
- Reporting vs. Analysis: "Analysis and reporting" features were dominant in Business Analyst (85.0%) and Data Analyst (70.7%) roles.
- Semantic Overlap: Figure 3 shows that Data Analysts and ML Engineers share a relatively high cosine similarity (0.65) in their tasks, likely due to shared data pipeline responsibilities, while Business Analysts and Software Engineers have the lowest overlap (0.38).
Limitations and Future Work
- Internship Bias: Because the dataset is ~98% internships, the findings might not perfectly generalize to mid-career or senior roles.
- Data Gaps: Technical outages led to missing data between May and August 2026, which may have obscured some seasonal hiring trends.
- LLM Dependency: The methodology relies heavily on the accuracy of GPT-6 for segmentation and verification. While the authors used a "held-out company" strategy to mitigate bias, the "LLM-judging-LLM" loop remains a point of scrutiny.
- Future Directions: The authors suggest expanding the dataset to include more full-time roles and tracking the same companies over multiple years to observe how specific roles (like "AI Engineer") emerge and stabilize.
Summary updated