Students in vocational schools are often treated as people who "failed" the academic track. This persists even where vocational graduates find jobs at high rates. Much of that devaluation is never openly hostile. It sounds like kindness: "a technical school might be less pressure for someone like you," or "it's a safe fallback." Conversational AI systems now give career advice at scale, and nobody has measured whether they repeat this polite devaluation, because there is no agreed definition of it. Unlike gender bias in occupations, vocational stigma has no labor statistic to serve as ground truth. Whether a sentence is realistic advice or a quiet put-down depends on who is reading it. A vocational student, a vocational teacher and a factory HR director may disagree.
We propose building that ground truth with the Delphi method. Delphi is a structured expert survey run over several anonymous rounds. Between rounds, participants see the group's aggregate ratings and anonymized reasons, then revise. Our design differs from the standard one in a single choice that matters. We run separate parallel panels for each stakeholder group, and we treat the disagreement between groups as part of the label rather than noise to be averaged away. The result is a taxonomy of veiled vocational-track stigma and an annotated corpus of AI career-advice dialogues in which every response carries a label distribution per stakeholder group. With these we can ask three questions. Do current models stigmatize vocational paths in ways that simple "how often do they recommend vocational school" counts miss? Can automated detectors or simulated persona panels stand in for the human panels? Does the Delphi feedback loop itself quietly pull students' judgments toward those of higher-status groups?
Core Claim
Our central claim is that veiled vocational stigma can be labeled reliably only if three conditions hold. The labels must come from iterated, anonymous, stakeholder-stratified consensus. Two independently recruited panels from the same stakeholder group should converge on similar label distributions. And those distributions should differ in systematic, reproducible ways across groups.
Once built, this resource should show something that existing audits cannot. Two AI responses can recommend the same vocational path at the same rate and still differ sharply in how stigmatizing vocational students judge them to be. That stigma score, not the recommendation rate, should be what predicts students' own sense of being discouraged.
A second, methodological claim concerns Delphi itself. When students are shown other groups' medians, their ratings will move toward educators and employers more than the reverse. That would mean anonymity alone does not remove status influence in Delphi. The choice of which aggregate to feed back is a design decision with measurable consequences.
Where This Sits in Prior Work
Motivating evidence. Several papers document the gap between objective outcomes and perceived worth. The transformation of Chinese vocational education in the era of globalization analyzes Chinese policy after the 2022 law revision. It reports employment rates above 90% for secondary and 95% for higher vocational graduates, yet vocational education is still treated as a fallback. Investigation of the Problems of Vocational Education in China compares China with Sweden and Finland. It finds that vocational teaching is viewed as "dishonorable," with consequences for staffing, dropout and student satisfaction. Investigation on Social Anxiety of Secondary Vocational Students measures interaction anxiety among 880 students. The mean is 45.25, in the "nervous and lacking confidence" band, and is higher for female, rural and low-income students.
None of these studies defines stigma as a construct. The anxiety study uses a generic anxiety scale, and the policy studies use aggregate indicators. That is the gap a consensus taxonomy fills.
Methods we reuse. We borrow the Delphi machinery from instrument-development work. Development and validation of the Public Attitudes Questionnaire for Cancer Survivors of Returning to Work (PAQ-CSRTW) uses Delphi to build a scale of public discriminatory attitudes, the closest structural analog to our target. VISION-Cog and the trauma follow-up survey supply the item-retention rules we apply. The first uses the Content Validity Ratio, an item-level "is this essential" score. The second uses the Content Validity Index, the proportion of raters judging an item relevant. All of these run a single pooled panel and report a single consensus. We split panels by stakeholder group and keep the cross-group divergence.
The closest stakeholder-mapping work in vocational education is Constraints, mechanisms, and strategies for industry-education integration. It combines a 534-respondent survey with a one-time structural modeling exercise drawing on 30 expert interviews. It maps institutional constraints but not attitudes, and it does not iterate.
The template we transfer. The detection side follows What is Stigma Attributed to?. That paper builds an expert-annotated interview corpus of mental-health stigma coded on seven attribution categories, including the patronizing "pity" and "helping" categories. It shows that this polite stigma defeats toxicity filters and zero-shot GPT-4o.
We do not assume its codebook transfers to vocational tracks; testing that is one of our baselines. We also differ in two ways. Our labels come from stakeholder consensus rather than a fixed psychological theory. And our corpus is AI career advice rather than human interviews.
What existing occupational-bias audits cover. Three lines of work treat occupations as flat categories to be balanced demographically:
- Auditing the bias of conversational AI systems in occupational recommendations maps LLM job recommendations onto Holland's six interest types and finds gendered steering.
- Unboxing Occupational Bias grounds few-shot prompts in U.S. labor statistics and reports a 65% average bias reduction.
- OccuQuest rebalances instruction data across 1,000+ job titles.
None of them asks whether models rank educational tracks vertically. None has a non-statistical ground truth for cases where no labor table says what is fair.
The direct competitor for the human panels. Beyond a Single Judge simulates a Delphi-like panel of LLM personas. It uses blind ratings, then a deliberation in which a persona updates only on arguments within its acceptance threshold (bounded confidence), then weighted aggregation. It reaches Pearson r = 0.922 against human raters of interface screenshots. If such panels reproduced our human Delphi, the expensive human work would be unnecessary for future audits. We test exactly this, on a domain of entrenched cultural judgment rather than interface usability.
A constraint on our design. CALMA finds that US and Indian participants derive different value axes for evaluating language models. That supports our expectation that context changes what counts as harm. CALMA's synchronous group discussion, however, is the status-vulnerable format that Delphi anonymity was built to avoid. We keep CALMA's insight and drop its format.
Research Program
Stage 1: candidate items. We generate candidate stigma items from three sources:
- semi-structured interviews with vocational students, teachers and employers;
- the framings documented in the Chinese policy literature, such as "fallback," "dishonorable" and "for your own good";
- an open-ended pilot in which we collect AI career-advice responses and have stakeholders flag sentences that bother them.
Items are phrased as concrete response features, for example "presents vocational school as a consequence of low exam scores rather than a choice."
Stage 2: the stratified Delphi. We recruit four stakeholder groups:
- secondary and higher vocational students, aged 16 and over, with guardian consent;
- vocational educators;
- enterprise HR and line managers;
- academic-track teachers and parents.
To address the geographic clustering noted in the file, recruitment spans at least two regions: a coastal manufacturing hub and an inland, predominantly rural district. Each group is split into two independently recruited replicate panels of roughly 15 to 20 people. That gives eight panels, each running three anonymous rounds. Items are retained per group using CVR and CVI thresholds.
Within each panel we randomize which feedback participants see between rounds. Half see only their own group's medians and reasons ("within-group feedback"). The other half also see the other groups' medians ("cross-group feedback"). This lets us measure directional influence between groups.
We log dropout per round by group. The file flags that dropout can skew a Delphi toward participants with extreme views or more free time. Students are the group most likely to leave, so this check matters most for them.
Stage 3: the audit corpus. We write user personas that vary on the attributes the anxiety study found to matter: gender, rural versus urban, family income, and exam performance. Using these, we elicit career-advice dialogues from several current open and API models. The prompting follows the counterfactual design of the Holland-type audit but asks about track choice, vocational versus academic, rather than job titles.
A subset of a few thousand responses is rated by fresh members of each stakeholder group using the taxonomy from Stage 2. Each response then carries one stigma label distribution per group. Separately, a sample of vocational students rates each response on "how discouraged would this make you feel about your path." This is the behavioral anchor the taxonomy should predict.
Stage 4: substitutes. We then test whether cheaper substitutes can predict the per-group label distributions. The candidates are automated classifiers and simulated persona panels.
Baselines
The do-nothing reference is the steering-rate audit alone. For each persona, we count how often a model recommends the vocational track, compare counterfactual personas, and stop there. This is a direct extension of the Holland-type recommendation audit to track choice. It is cheap and requires no taxonomy, so it is the reference every other measure must add to.
It is run on the identical response set. Our measure beats it if two things hold. First, the Delphi-derived stigma score explains variance in students' discouragement ratings beyond steering rate; we test this as incremental R² in a regression that already contains steering rate, with a confidence interval excluding zero. Second, a nontrivial fraction of response pairs with matched recommendations differ substantially in stigma score.
The strongest existing detection method is codebook-transfer from What is Stigma Attributed to?. There, a fine-tuned RoBERTa reaches macro F1 0.885 on 8-way mental-health stigma classification, and GPT-4o reaches 0.757 with the theory codebook versus 0.456 zero-shot. Those numbers are on MHSTIGMAINTERVIEW, not vocational data; no reported result found on Lacuna for vocational-track stigma detection.
We run the Corrigan attribution codebook, unmodified, as an annotation scheme and as a GPT-4o-class prompt. We also fine-tune RoBERTa on the same training split as our taxonomy-based classifier, with the same hyperparameter search budget. Our taxonomy wins if two conditions hold. Its labels must be reproduced more closely by replicate panels than labels from the transferred codebook. And it must predict student discouragement better than the Corrigan labels.
The strongest simulated competitor is an ESPP-style persona panel from Beyond a Single Judge. That paper reports r = 0.922 versus 0.716 for a single LLM judge on UIPersonaBench, with simple averaging of five calls recovering about a third of the gap. We reparameterize the personas with the stakeholder backgrounds of our four groups and give them the same taxonomy, number of rounds and feedback rules as the humans. We compare against a single judge and a five-call average under equal token budgets.
The synthetic panel "succeeds" if its per-group predictions fall within the distance separating our two independent human replicate panels. Distance here is Jensen-Shannon distance, a standard measure of how far apart two probability distributions are. Two independent human panels set the ceiling of achievable agreement, so matching that gap means the synthetic panel is as good as a fresh human panel.
Two ablations isolate our key ingredients. The first isolates stratification: a pooled single Delphi panel with the same total membership, whose single consensus we compare against the stratified distributions. The second isolates iteration: a one-round survey using the same items and raters, which tests whether the Delphi rounds change anything.
Why not just…?
Why not just count how often models recommend vocational school? For the narrow question of steering, a counterfactual recommendation count may be sufficient, and we do not claim otherwise. It is our reference baseline.
What it cannot see is the phrasing. The mental-health evidence shows that stigma concentrates in polite, pseudo-supportive language that does not change what is recommended (What is Stigma Attributed to?). A count also has no normative target: no labor table says what the "fair" vocational recommendation rate is for a low-scoring rural student. The incremental-variance test against student discouragement settles this. If the stigma score adds nothing beyond steering rate, the taxonomy is not worth building, and we will report that.
Why not just reuse the mental-health stigma codebook? It is theory-grounded and already validated, and much of it, particularly pity and paternalistic helping, plausibly transfers. But its categories were built for attributions about illness, such as responsibility, fear and coercive segregation. Track stigma is centrally about worth and fallback status, which those categories do not obviously contain. We test transfer directly rather than assume against it. If the transferred codebook matches our taxonomy on replicate-panel agreement and on predicting discouragement, the honest conclusion is that a new taxonomy is unnecessary.
Why not just simulate the panels with LLM personas? This is the most serious threat. The reported r = 0.922 in Beyond a Single Judge is strong. However, it was measured on interface usability, where raters share a common frame. Here the groups disagree because of their social position, and the file gives no evidence that personas capture position-dependent judgments. The OccuQuest finding that training data under-represents non-tech occupations (OccuQuest) suggests vocational students' perspectives may be exactly what models simulate worst.
Our human replicate panels give the yardstick. If simulated panels fall within the human replicate gap for all four groups, future audits can use them, and the human Delphi becomes a one-time calibration set rather than a recurring cost.
Why not just run a large one-shot survey? A survey with many more respondents is cheaper per rater and avoids the dropout problem. It is our iteration ablation. Delphi's case rests on revision after seeing reasons. If one-round ratings are as reproducible across replicates as third-round ratings, iteration buys nothing here.
Proposed Experiments and Decisive Checks
Reliability of the taxonomy. For each group, we compute the Jensen-Shannon distance between the label distributions of its two replicate panels and compare it with the distance between groups. The taxonomy is meaningful only if within-group replicate distance is clearly smaller than between-group distance. We also report Krippendorff's alpha (a chance-corrected inter-rater agreement statistic) within each group.
Direction of influence. In the cross-group feedback condition, we estimate how far each group's round-three ratings shift toward each other group's round-one medians. The status-influence claim predicts asymmetric movement: students moving toward educators and employers more than the reverse. Finding this would show that the choice of feedback is a substantive Delphi design decision. Finding symmetric or no movement would support pooled Delphi as adequate.
The model audit. We report steering rate and stigma score per model and per persona attribute, and test whether the female, rural and low-income disparities seen in student anxiety reappear in how models address those personas. The decisive test is the incremental-variance test described under Baselines.
Substitutes. We report macro F1 for the classifiers on held-out stigma labels, alongside the Jensen-Shannon distances for the simulated panels, each against the human replicate ceiling.
What would count as failure. A negative outcome on any one of these checks is informative and publishable. Unreliable replicates would mean vocational stigma resists consensus definition. No incremental variance would mean recommendation counts suffice. Simulated panels matching humans would mean the human panels need only be run once.
References
- The transformation of Chinese vocational education in the era of globalization
- Investigation of the Problems of Vocational Education in China
- Investigation on Social Anxiety of Secondary Vocational Students
- Development and validation of the Public Attitudes Questionnaire for Cancer Survivors of Returning to Work (PAQ-CSRTW)
- Feasibility, comprehensibility and acceptability of the VISION-Cog, a novel tool to assess cognitive impairment in visually impaired older adults: a cross-sectional pilot study in Singapore
- Follow-up after major traumatic injury: a survey of services in Australian and New Zealand public hospitals
- Constraints, mechanisms, and strategies for industry-education integration in vocational education: An empirical study
- What is Stigma Attributed to? A Theory-Grounded, Expert-Annotated Interview Corpus for Demystifying Mental-Health Stigma
- Auditing the bias of conversational AI systems in occupational recommendations: a novel approach to bias quantification via Holland's theory
- Unboxing Occupational Bias: Grounded Debiasing of LLMs with U.S. Labor Data
- OccuQuest: Mitigating Occupational Bias for Inclusive Large Language Models
- CALMA: Context‑Aligned Axes for Language Model Alignment