Scoring Cultural Abstraction by Tracing Designs to Their Sources

Good cultural design often removes detail on purpose. A designer who turns a Tang-dynasty collar into a single curved line on a modern jacket, or a circular "group flower" costume motif into a logo, keeps the motif's structure and drops its literal surface. The automated metrics now used to judge cultural imagery treat that removal as a failure. They reward images that match keywords, which favors stereotyped exaggeration, or they penalize any departure from realistic reference photographs. This matters for machines and for people. Generative models are now judged on "cultural transcreation," meaning the adaptation of a heritage artifact into a modern design. Language educators want to measure whether students can express their own culture to foreign audiences instead of only absorbing the target culture. Neither group has an automatic way to tell a faithful abstraction from a cheap stereotype.

We propose to score an abstracted design by whether it can be traced back to its source. We show the design to several independent vision-language models, which are systems that answer questions about images, and to international human viewers. Each is asked to pick the heritage artifact the design came from, out of a set of deliberately confusable candidates. A faithful abstraction stays traceable even when its surface detail is gone. A stereotype stays traceable only to a generic national cliché and not to the specific artifact. A design with foreign or hallucinated elements pulls viewers toward the wrong candidates. We first test the metric on a controlled ladder of abstraction levels and corruptions built from an existing heritage benchmark. We then use it as a measure of a soft skill: a student's ability to export their own culture through a design plus a short English explanation. We compare it against the survey and exam-score measures the intercultural-education literature currently relies on.

Core Claim

Asking "which specific source is this derived from, among near-misses" separates intentional abstraction from stereotyping better than asking "is this culturally faithful." Whole-image faithfulness judgments mix up two things. One is how much literal detail survives. The other is whether the motif's defining structure survives. Contrastive tracing against hard distractors measures only the second. The claim fails if a strong model judge that is shown the source image can already rank these designs as well as experts do. We test for that directly.

Where This Sits in Prior Work

The closest metric work supplies both the motivation and a method to reuse. Culture in Action introduced the CULTIVate benchmark and the AHEaD decomposition, which scores an image separately for alignment, hallucination and exaggeration and then combines them into a FAITH score. It showed that CLIPScore, a standard image-text similarity measure, correlates positively with stereotyped exaggeration. AHEaD's decomposition is the right instinct, and we borrow its separation of "missing required elements" from "inserted foreign elements." Its exaggeration term, however, counts reduction and traditional iconography as errors against realistic reference photographs. That makes it the wrong instrument for design work where literal detail is discarded on purpose.

Hanfu-Bench supplies our source material and our target task. It provides expert-curated historical garments annotated by structural element (sleeve, collar, the front overlap called jin, bottoms, outerwear, period). It also defines a transcreation task in which the best generative model reached only a 42% success rate under human evaluation. Transcreation is graded entirely by hand there. We aim to replace that manual step with an automatic score that agrees with experts. Hanfu-Bench's element annotations also give us a ready-made vocabulary of the "structural primitives" that design studies such as the Huangmeixi opera product study and the Florasis·Miao Impressions case describe only qualitatively. Those studies decompose motifs into points, lines, colors and circular structures, but they are validated only by consumer surveys. They motivate the problem and give no measurement.

Multimodal Cultural Safety (CROSS) constrains our design. Its rubric checks awareness, explanation, compliance and helpfulness, which amounts to checking whether a model avoids symbolic harm. That is reactive. It does not score whether something culturally meaningful was successfully expressed. We keep a harm check as a veto, but the positive score has to come from elsewhere.

On the human side, Enhancing intercultural competence in technical higher education quantifies intercultural competence from 1,050 questionnaire responses using fuzzy scoring and clustering. It found that CET-4 scores above 600 did not imply cultural knowledge, and it reported a mean competence of 0.4952 on a 0–1 scale. This finding motivates us: exam scores miss the construct. It also sets the baseline we must beat, because the measurement is still a self-report survey. The Importance of Chinese Cultural Infiltration in College English Teaching names the target failure "cultural aphasia," meaning fluent learners who cannot explain their own culture. The 4T Lenses argues that students should express culture across media, including images and generated video, but offers no measurement. A Scoping Review of Computational Thinking Assessments documents a broader pattern: few non-cognitive assessments are validated, and only two are automated. Our human study is a small, specific answer to that gap, not a general soft-skill instrument.

Research Program

Step 1: an abstraction ladder. We take about 150 Hanfu-Bench garment sets spread across dynasties. For each, trained design students produce four variants under written instructions. The first is a near-literal modern rendering. The second is a morphological abstraction that keeps the annotated structural elements but simplifies them. The third is an atmospheric abstraction that keeps palette, silhouette and mood while dropping most explicit structure. The fourth is a free design by the same designer.

We also produce three corrupted variants per set. One is anachronistic, mixing elements from another dynasty. One inserts a foreign-culture element. One is a crude stereotype that replaces the specific garment with a generic "Chinese costume" cliché, generated with a text-to-image model and filtered by hand. Native experts rate every variant for faithfulness to the specific source and for design quality. International viewers rate legibility.

This gives us pairs where the correct ordering is known by construction. The central case is a morphological abstraction, which should score high, against a stereotype of the same garment, which should score low.

Step 2: the tracing score. For a design $d$ with source $s$, we build a candidate set $C(s)$ of $K$ artifacts. Hard distractors are drawn from three pools. The first pool is the same dynasty with a different garment type. The second is a neighboring dynasty with the same garment type, using Hanfu-Bench's period labels. The third is garments from other cultures with similar geometry, which we collect and license separately. A panel of $M$ different vision-language models (open and proprietary) is shown $d$ and $C(s)$ and asked to choose the source. The tracing score is

$$T(d) = \frac{1}{M}\sum_{m=1}^{M} \log \frac{p_m(s \mid d, C(s))}{1/K},$$

where $p_m(s \mid d, C(s))$ is model $m$'s probability for the true source and $1/K$ is chance. $T$ is zero at chance and positive when the source is recoverable. We average over several random candidate orderings to remove position bias.

Two further components complete the score. The first is element-level tracing. We repeat the choice with the candidate set restricted to distractors that differ from $s$ in only one annotated element, such as a different collar. This shows which primitives the design actually carried. The second is a foreign-element veto borrowed from AHEaD's hallucination term. It flags designs whose probability mass shifts toward other-culture distractors.

Using a panel of models and hard distractors guards against one model's shortcut. It also addresses the prompt-dependence documented in Rethinking AI Cultural Evaluation, where outputs shifted across prompting frames and over 40% of Chinese-scenario outputs were unclassifiable. A forced choice among candidates cannot collapse into evasive neutrality.

Step 3: from metric to soft-skill measure. In college English courses, students receive a heritage motif. They produce a modern design, either drawn or made with generative tools, and a 100–150-word English explanation aimed at a foreign audience. We score the design alone, the explanation alone (a text-only tracing variant), and the two together. International readers who have never seen the source perform the same forced choice, so we have a human tracing score alongside the model score.

The quantity of interest for cultural aphasia is whether the explanation adds traceability and correct meaning beyond the design. A fluent student who cannot articulate the motif's meaning should show high linguistic quality but little tracing gain from their text. This operationalizes the aphasia construct as an output measure rather than a self-report.

Baselines

The naive reference is CLIPScore between the design and the source's expert caption. Culture in Action reports that this measure rewards exaggeration, so we expect it to prefer stereotypes over abstractions on the ladder pairs. It is run on identical images at no tuning cost. Beating it means higher pairwise agreement with expert orderings, and we expect it to fall at or below chance on abstraction-versus-stereotype pairs.

The strongest existing automatic metric is AHEaD/FAITH. On CULTIVate it achieved a 27% higher correlation with human faithfulness judgments than MLLM-as-judge baselines (Culture in Action). We run it with its published components on our ladder, using the Hanfu-Bench source photographs as its reference images. We also run a variant with the exaggeration term removed, so the comparison does not rest on an obviously mismatched term. No reported result was found on Lacuna for any metric's agreement with experts on intentional abstraction, so the comparison is new. Beating AHEaD means higher Spearman correlation with expert faithfulness ratings over the whole ladder, and higher pairwise accuracy on abstraction-versus-stereotype pairs, with bootstrap confidence intervals that do not overlap.

The most dangerous baseline is a rubric-prompted model judge. The same model panel receives the design, the source image, and a written rubric covering primitive retention, anachronism, foreign insertion and stereotype. It returns a 1–10 score. This matches the evaluator's model access and prompt-tuning budget, which is ten prompt iterations on a held-out development split for both it and our method. It differs from our score only in using absolute grading rather than contrastive tracing.

The key ablation is our score with random distractors in place of hard ones. It isolates whether the confusable candidate set is doing the work.

For the human study, the baselines are CET-4 scores and the fuzzy-scored questionnaire of Enhancing intercultural competence, administered to the same students. Success means our score predicts an independent expert-graded live task better than both, with prediction measured as incremental variance explained.

Why not just…?

Why not just use a rubric-prompted model judge that sees the source? This is the real threat. If it matches our score, the contribution reduces to an elaborate prompt. The evidence on its side is limited. The FAITH comparison in Culture in Action shows that plain model-as-judge scoring lags a decomposed metric on realistic images. Hanfu-Bench shows that models do notably worse on multi-image comparative questions than on single-image ones, which cuts in both directions for us, since tracing is itself a multi-image comparison. Our experiment settles the question directly. If the rubric judge's pairwise accuracy on abstraction-versus-stereotype pairs falls within the confidence interval of ours, we will report that absolute judging suffices and drop the claim. We will then keep only the element-level diagnostics as a contribution.

Why not just retune AHEaD by dropping or reweighting its exaggeration term? AHEaD still measures against realistic reference images. Removing exaggeration leaves an alignment term that counts surviving elements. That term should also rate a heavily detailed stereotype above a sparse but precise abstraction. The ablated-AHEaD baseline tests exactly this. If it closes the gap, the simpler fix wins and we will say so.

Why not just use CLIPScore or embedding similarity to the source? Similarity measures reward surface overlap, and Culture in Action found CLIPScore positively correlated with exaggeration. Our ladder includes the exact case, a detailed stereotype against a spare abstraction, on which we predict this measure orders the pair wrongly.

Why not just measure students with language exams or intercultural questionnaires? Enhancing intercultural competence already shows that CET-4 above 600 does not track cultural knowledge. Its own questionnaire is self-reported and never observes whether a student can produce culturally legible output. Our incremental-validity test measures whether artifact tracing adds predictive power beyond both on a live explanation task graded by experts blind to all scores. If it adds none, the soft-skill claim fails.

Proposed Experiments and Decisive Checks

The first decisive check runs on the abstraction ladder. We report pairwise accuracy against expert orderings, separately for abstraction-versus-literal, abstraction-versus-stereotype, and abstraction-versus-anachronism pairs, and Spearman correlation with expert faithfulness overall. The prediction is that our score and the rubric judge both handle anachronism. Only a contrastive, distractor-based score should order abstractions above stereotypes. CLIPScore and AHEaD should invert or tie on those pairs.

The second check tests transfer to model outputs. We take generated transcreations produced by Hanfu-Bench's caption-refine-generate pipeline, have experts grade them with the Hanfu-Bench protocol, and test whether our score reproduces those grades without new human labels. The reference point is the reported 42% human-evaluated success rate.

The third check is a guardrail that confirms we did not break stereotype detection. On a CULTIVate subset, our foreign-element veto should flag hallucinated insertions about as well as AHEaD's hallucination term. If it does worse, the metric trades one failure for another.

The fourth check is the human study, with about 200 students across two institutions. Measures are collected before and after one semester in which half the sections use the design-plus-explanation task. We test incremental validity over CET-4 and the questionnaire. We also test whether the explanation's tracing gain grows with instruction, which is the direct signature of reduced cultural aphasia. The study is limited to Chinese heritage and Chinese learners, which is the same single-context limitation that affects the cited survey work. Extending distractor pools and viewer panels to a second heritage tradition is the natural next step, and it stays out of this study's scope.