Courses that teach machine learning students to find algorithmic bias almost always use social data: redlining records, hiring tables, face images, dialect prompts. Those datasets matter, but they make two core technical skills hard to learn. The first is telling a real disparity between subgroups apart from noise in a small test set. The second is refusing to credit a "debiasing" method until it has beaten a properly tuned baseline. In social data the true cause of a disparity is contested and usually cannot be checked, so students have no ground truth to calibrate against. This proposal tests whether students learn these two skills better when their first audit is of a model that predicts the strength of fiber-reinforced concrete. In that setting, some subgroup failures have a known physical cause that can be verified, and others are pure small-sample noise. The proposal then asks whether those skills transfer to a later audit of a social system.
The materials side supplies more than a dataset. Composite-materials research has a mature habit of asking whether two additives combined do better than the sum of their separate effects. It uses a simple ratio, the synergy index, which classifies a combination as superadditive (greater than one), additive, or subadditive. Ethics-education research has no equivalent. Studies add instructional components one at a time and report pre/post gains on cohorts of 14 to 42 students. We borrow the index as the summary statistic of a factorial classroom trial. It tells us whether a materials audit followed by a social audit beats either alone, or whether the second module merely adds load.
Core Claim
An auditing exercise is most instructive when the student can find out afterward which apparent disparities were real. Hybrid-fiber concrete gives exactly this. Z. Zang et al. show that predictive models with a hybrid-effect coefficient are accurate at low fiber dosages but break down at high dosages. The reason is physical: fibers clump ("balling"), voids form, and dispersion becomes uneven. A regressor trained mostly on low-dosage mixes will therefore show a real, mechanistically explained error gap in the high-dosage subgroup. Thinly sampled fiber combinations will show gaps that are noise. We claim that auditing such a model teaches the noise-versus-real distinction and the tuned-baseline habit better than an equally long audit on social data or on a non-human synthetic benchmark like Waterbirds. We also claim these gains show up on a held-out social audit. If the claim fails, and the gains do not transfer or Waterbirds does just as well, then the materials framing is a curiosity and should be dropped.
Prior Work and Positioning
Three lines of educational work motivate the gap. Kolisko and Anderson had undergraduates build minimal-pair probes for foundation models. They report that students struggled to separate systematic bias from sampling noise and that the topics could be emotionally taxing for marginalized students. Both problems are things our design targets directly. McClure et al.'s SourceML had high schoolers train text classifiers on 1930s redlining descriptions and find proxy words like "wooded." They report a significant pre/post gain ($U = 462, p < 0.05$, N=42). Krakowski et al.'s TechHive AI organizes instruction into determination, data, design, and decision stages, and reports only qualitative outcomes. All three use social data, single cohorts, and no randomized comparison of instructional domains. We keep their audit structure and change the domain, the comparison, and the outcome measure.
Two benchmarking papers constrain what "correct" student behavior means. Tan et al.'s NH-Fair spent over 10,000 GPU hours showing three things. A thoroughly tuned standard training run often matches specialized debiasing methods. Fairness regularizers such as GapReg often reach parity by lowering the advantaged group's accuracy. Composite augmentation improves both accuracy and parity. This is the lesson our transfer test checks for, and it contradicts the celebratory framing of debiasing re-implementation in Lučić et al.'s reproducibility course (Reproducibility as a Mechanism for Teaching Fairness, Accountability, Confidentiality, and Transparency in Artificial Intelligence). Correct-N-Contrast supplies a concrete, strong method students will be asked to evaluate. It raises CelebA worst-group accuracy from 81.1% to 88.8%.
The materials literature supplies the method and the data. Zang et al. define the synergy index and document the high-dosage breakdown. Chang Gao et al. and Fengyuan Yang et al. report additional hybrid systems with known non-monotonic dosage effects. Yang et al. find activator synergy alongside pore disproportionation at low density. No paper in the file combines physical-materials data with ethics instruction, and the closest bridge, TechHive's staged framework, never quantifies whether combined components add up.
Research Program
The first step is building the teaching dataset. We compile mix designs and measured compressive and flexural strengths from the hybrid-fiber, foamed-concrete, strain-hardening, and rubberized-concrete studies in the file and their reported tables. Each row records fiber types, volume fractions, curing age, and source lab. The usable size is unknown until compiled, and we expect at most a few hundred rows. We deliberately make the training split rich in low-dosage mixes. We seed the dataset with three kinds of subgroup gap, each documented in an instructor key. The first is a real high-dosage gap with a physical explanation from the source papers. The second is a confound tied to the source lab: one lab's specimens are systematically stronger, a stand-in for a proxy variable. The third is several noise-only gaps in subgroups with five or fewer specimens. Students also receive a "mitigation" that equalizes error across dosage bands by inflating low-dosage error. This is the materials analog of NH-Fair's "fairness by harm." Students also receive the option of requesting additional high-dosage specimens. That option is the data-side fix, and it mirrors TechHive's point that the right fix is sometimes not algorithmic.
The second step is the trial. In a large introductory machine learning course, every student gets the same lecture on permutation tests, confidence intervals for subgroup metrics, and tuned baselines. Lab sections are then randomized to a two-hour hands-on audit in one of five conditions. The conditions are: no hands-on audit (extra lecture practice only), a social audit using the TechHive hiring data, a materials audit, both audits in sequence with total time held fixed, and a Waterbirds audit. The Waterbirds condition isolates "non-human but synthetic" from "non-human with checkable mechanism." The target is several hundred students across at least two institutions. We pre-register analyses at the section level to avoid the single-cohort, self-selected samples criticized throughout this literature.
The synergy index is defined on transfer gains over the no-audit condition. Write $\Delta_s$, $\Delta_m$, and $\Delta_{sm}$ for the mean gain in the social-only, materials-only, and combined conditions:
$$S = \frac{\Delta_{sm}}{\Delta_s + \Delta_m}.$$
$S$ is just a rescaled interaction contrast from the factorial design. We report it with bootstrap intervals because "superadditive versus subadditive" is the question curriculum designers actually ask. Because the combined arm halves each module's time, $S < 1$ is the null expectation, and finding $S \geq 1$ would be informative.
Baselines
The naive reference point is the lecture-only condition. Students receive the identical statistics and tuned-baseline content but no hands-on audit. Every other arm must beat it, or hands-on auditing adds nothing beyond telling students the rules. It receives equal contact time through worked examples.
The strongest existing pedagogical competitor is the social-data audit modeled on TechHive AI and SourceML. It is the established practice, and its domain matches the transfer task, so it has a home-field advantage. It runs with the same instructors, duration, statistics primer, and grading rubric as the materials arm. No reported transfer-score result was found on Lacuna for it. The only quantitative outcome in the file is SourceML's within-cohort $U = 462, p < 0.05$, which is not comparable to a randomized transfer measure. The materials arm beats it only if its transfer score exceeds the social arm's on the held-out social audit, with a pre-registered minimum effect of 0.3 standard deviations.
The Waterbirds audit is the ablation that isolates the key ingredient: a disparity whose cause can be verified. Waterbirds, used in Correct-N-Contrast's evaluation, is non-human and emotionally neutral, and its spurious background correlation is known by construction. If it matches the materials arm, then mechanistic, real-world grounding adds nothing beyond neutrality and a known answer.
Within the transfer task, the technical baselines students should apply are a tuned standard training run versus Correct-N-Contrast on CelebA. For the strongest method we have CNC's reported 88.8% worst-group accuracy on CelebA. For tuned ERM, no reported number was found on Lacuna for NH-Fair's tuned ERM on CelebA. We will compute it ourselves under a fixed sweep before the trial, so the answer key does not depend on unverified figures.
Why not just…?
Why not just use Waterbirds or colored MNIST, which are already non-human and have known spurious correlations? This is the most serious threat. It is why Waterbirds is its own arm rather than an afterthought. Our bet is that Waterbirds has a known answer but no mechanism students can reason about. The bird-background link is an artifact of construction. In concrete, the high-dosage gap is explained by fiber clumping that students can read about in the source studies. It also coexists with noise-only gaps, which Waterbirds lacks. If the Waterbirds arm matches the materials arm on noise-discrimination and baseline-discipline scores, we conclude the cheaper benchmark suffices and recommend it.
Why not just teach hypothesis testing and tuned baselines in lecture? Every arm already gets that lecture, and the lecture-only arm measures exactly this alternative. Kolisko and Anderson's observation is that students who were told to be careful still could not separate noise from bias in practice. If the lecture-only arm performs as well as the hands-on arms, the hands-on audit is unnecessary and the proposal's premise fails.
Why not just stay with social data, since that is what students will audit after graduation? The social arm tests this directly and has the advantage on a social transfer task. The case against it rests on calibration: students cannot learn when a disparity is real if no instance ever reveals the truth. It also rests on the emotional burden Kolisko and Anderson report. We measure that burden with a short affect survey in every arm. If the social arm wins on transfer, the right curriculum is social-first, and we will report it as such.
Why not use the synergy index only as a metaphor and skip the factorial? Without the both-modules arm, we cannot tell curriculum designers whether stacking modules helps or simply crowds the time. Gap analyses of documentation-checklist fatigue in Heger et al. suggest crowding is a real risk. The factorial costs one extra arm.
Proposed Experiments
The primary outcome is a held-out, individually completed social audit given one week after the lab. Students receive a classifier trained on Adult or on a salary-advice probe in the style of Sorokovikova et al., along with a list of six apparent subgroup gaps. The instructor key marks three gaps as real and three as noise. Students also receive a claimed mitigation, a regularizer that achieves parity by degrading the advantaged group. We score three things. The first is accuracy at classifying gaps as real or noise, summarized as the area under the curve of students' stated confidence. The second is whether the student compares the mitigation against a tuned baseline and flags the accuracy loss for the advantaged group. The third is the quality of the written justification, scored by blinded raters.
Secondary outcomes are the affect survey, time-on-task, and a three-month delayed retest in a follow-on course where possible.
The decisive checks are three. First, the materials arm must beat the lecture-only arm on gap discrimination. Second, it must beat or match the social arm on transfer. Third, it must beat the Waterbirds arm, which is what separates "checkable mechanism" from mere neutrality. We also report $S$ for the combined arm. A null result on the third check, with success on the first, would still be useful. It would say that the active ingredient is a known answer, not concrete.
References
- Research on mechanical properties and prediction methods of hybrid fiber concrete for airport pavements
- Exploring Social Biases of Large Language Models in a College Artificial Intelligence Course
- Modeling with Primary Sources: An Approach to Teach Data Bias for Artificial Intelligence and Machine Learning Education
- Authentic Integration of Ethics and AI through Sociotechnical, Problem-Based Learning
- Benchmarking Bias Mitigation Toward Fairness Without Harm from Vision to LVLMs
- Reproducibility as a Mechanism for Teaching Fairness, Accountability, Confidentiality, and Transparency in Artificial Intelligence
- Correct-N-Contrast: a Contrastive Approach for Improving Robustness to Spurious Correlations
- A Review on Multi-Scale Toughening and Regulating Methods for Modern Concrete: From Toughening Theory to Practical Engineering Application
- Pore Structure, Hardened Performance and Sandwich Wallboard Application of Construction and Demolition Waste Residue Soil Recycled Foamed Concrete
- Understanding Machine Learning Practitioners' Data Documentation Perceptions, Needs, Challenges, and Desiderata
- Surface Fairness, Deep Bias: A Comparative Study of Bias in Language Models