When Tutors Drift Home: Coupled Pedagogical and Cultural Reversion in Multi-Turn LLM Tutoring

Language models used as tutors fail in two ways that have so far been studied separately. The first is pedagogical: a model told to guide a student with hints gradually stops doing so and starts handing out answers. In one study, a tutor trained only by imitation fell from 75% to 60% hint-giving accuracy over eight turns (Pedagogical Alignment of Large Language Models). The second is cultural: a model told to speak from a non-Western context drifts back toward Western defaults as a conversation goes on. It switches to Western names, foods, currencies and examples (contextual reversion, reported in work indexed under this author). This proposal tests whether these are two faces of one failure. Our hypothesis is that, deep into a conversation, the model loses its grip on its instructions and falls back to its strongest training prior. For a tutor, that prior is a generic, answer-giving, Western-flavored explainer. If so, the two drifts should happen in the same turns, and one training signal aimed at late-turn behavior should repair both.

The practical stakes are concrete. An Arabic-speaking student working through a word problem may get Socratic hints built around local markets and Egyptian pounds in turn two. By turn nine the same student may be getting a worked solution about a baseball team's budget in dollars. Current tutoring alignment trains on English, US-textbook dialogues generated by GPT-4. Current cultural alignment evaluates single-turn fill-in-the-blank or question-answer items. Neither looks at the long-conversation regime where both failures appear.

Core Claim

We make two claims. The first is empirical: pedagogical decay and cultural reversion in multi-turn tutoring are positively coupled at the level of individual turns. A turn in which the tutor abandons scaffolding is more likely to also abandon the learner's cultural context, beyond what turn depth alone predicts. The second is methodological: preference pairs built from the model's own late-turn failures can fix both at once. These pairs reject responses that drift in either respect. Training on them works only if it is paired with a calibration term that keeps the model from inventing cultural content when the learner gives no cue.

To make "coupled" testable, let $S_t$ be 1 if the tutor's turn $t$ gives an unprompted direct answer and 0 otherwise. Let $W_t$ be 1 if a turn that contains a culture-substitutable slot (a name, food, currency, place, or institution in an example or analogy) fills that slot with a Western entity, in a dialogue whose opening carries a clear non-Western cue. We fit

$$\text{logit}\, P(W_t = 1) = \alpha + \beta_1 \log t + \beta_2 S_t + u_{\text{dialogue}},$$

where $u_{\text{dialogue}}$ is a per-dialogue random effect. The coupling hypothesis is $\beta_2 > 0$ with depth controlled. If $\beta_2$ is near zero, the two failures are independent, and the right remedy is two separate fixes rather than one joint objective. That outcome would still be worth reporting.

We deliberately do not claim that Socratic scaffolding is culturally wrong for some learners. The research file contains a tempting version of that claim. Its only direct evidence is that preference for Socratic versus direct coaching splits by teaching experience, not culture: junior instructors preferred Socratic and those with more than three years preferred narrative (Exploring Conversational Design Choices in LLMs for Pedagogical Purposes). Hard-coding a mapping from culture to pedagogy from Hofstede-style dimensions would repeat the essentialism risk that CALM itself acknowledges. We therefore treat the instructed pedagogical style as the target and measure whether the model keeps to it. We include only a small exploratory learner-preference study.

How This Relates to Prior Work

Three papers motivate the proposal by documenting the separate failures. Pedagogical Alignment of Large Language Models frames tutoring as preference learning, using DPO, IPO and KTO on GPT-4-generated OpenStax biology dialogues. It shows that preference training holds 75% hint accuracy across eight rounds where imitation training decays. All of its data is English and US-curricular, and it never measures what content the hints use. Having Beer after Prayer? introduces the CAMeL benchmark of Arabic prompts and culture-specific entities. It finds a 40% to 65% preference for Western entities even in prompts that explicitly reference Arab culture, and traces part of this to Arabic Wikipedia being largely translated from Western articles. It studies single completions, not dialogue. The contextual-reversion work shows persona prompts fading over turns, but outside tutoring.

Two papers supply methods we reuse. CoCoA trains LoRA adapters with three losses: one favoring local entities when cultural cues are present, one keeping the model neutral when cues are absent, and one limiting drift from the base model. It shows that uniform debiasing flattens legitimate cultural associations. We borrow its neutral-calibration idea. We differ by moving from fill-in-the-blank tokens to generated multi-turn dialogue, and by treating turn depth as a central variable rather than a binary cue. CARE shows that a modest native-speaker preference set improves cultural scores by up to 29% over baselines trained on 3–17x more generic data. This supports our choice of native-authored seed material over translation.

Two papers constrain the design. NormAd finds that models hallucinate cultural significance in neutral scenarios, reaching only 42% accuracy there against 98% for humans. A naive "always localize" objective would therefore make things worse, which is why neutral calibration is mandatory in our design. CAReDiO notes that synthetic data inherits the generator's cultural ceiling. That rules out building our localized dialogues entirely with GPT-4.

Strategize Before Teaching jointly predicts a teaching strategy and the response. On TSCC it reports 5.47 BLEU versus 2.44 for an mBART baseline. We use its strategy labels from CIMA, TSCC and TalkMoves to define what counts as scaffolding. Its fixed taxonomy and single-turn targets do not address drift.

Research Program

The first component is a diagnostic testbed. Native-speaking teachers in Arabic and in two Camellia languages, for example Hindi and Korean, write seed tutoring problems in math and biology. Each problem embeds two to four culture-substitutable slots drawn from CAMeL and Camellia entity categories. Writing seeds natively rather than translating OpenStax avoids importing the Wikipedia trap into the test set.

From each seed we build two versions. The cued version has a clear local opening, such as the learner's name, city, and a local reference. The neutral version strips cultural markers. We then run tutoring dialogues to 12 turns with a simulated student whose behavior is scripted to apply pressure: wrong attempts, requests for the answer, and topic shifts. This kind of friction drives human teachers back to lecturing (Student teachers' interactional language). All dialogues are conducted in the target language, not English, which answers the English-centricity criticism raised against NormAd and CAReDiO.

Two measurements are taken per turn. Each slot filled by the tutor is labeled Western, local, or neutral using the CAMeL and Camellia entity lists, with LLM fallback adjudicated by native annotators on a sample. Each tutor turn is labeled for scaffolding using the CLASS-style judging of the Pedagogical Alignment paper, cross-checked against strategy labels on CIMA, TSCC and TalkMoves.

The second component is turn-deep, on-policy preference construction. We sample the current tutor through the testbed. At turns 6–12 we collect responses that drift pedagogically, culturally, or both. We pair each with a corrected response that keeps the scaffolding and local grounding while preserving the remaining content. The corrected response is written first by a strong model and then edited by native annotators, which keeps us below the synthetic ceiling CAReDiO describes.

Training uses DPO on these pairs, weighted toward deep turns. It adds a CoCoA-style neutral-calibration term computed on the neutral-version dialogues, which penalizes inventing local or Western specifics when no cue exists. The distinctive ingredient is that rejected responses come from the model's own late-turn failures under friction, not from single-turn contrasts written in advance.

Baselines

The naive reference point is the instruction-tuned base model, for example Llama-3.1-8B-Instruct as used in the Pedagogical Alignment paper. It receives a single system prompt naming the learner's context and asking for hints. Every method is compared against this model's per-turn decay curves for $S_t$ and $W_t$.

The persona-prompt baseline adds a detailed regional tutor persona and pedagogical instructions to that same model. This is the cheapest intervention a practitioner would try. Prior work suggests it decays, but that has not been shown in tutoring.

The strongest pedagogical competitor is the learning-from-human-preferences DPO recipe of Pedagogical Alignment of Large Language Models. That paper reports Llama SFT dropping from 75% to 60% over eight rounds while the preference-trained model maintained 75% accuracy, and a 13.1% accuracy gain for Llama-3.1-8B. We retrain it on the same base model, with the same number of preference pairs and the same hyperparameter search budget as our method. One version uses its original English OpenStax data. A second version uses pairs built from our seeds but restricted to turn 1 to 3 contrasts.

The strongest cultural competitor is CoCoA. It reports reducing the grounded cultural bias score on CAMeL and Camellia from 43 to 24, with a neutral score of 50.2 and at most a 0.38-point drop on ARC-Challenge. We train it on its own objectives with matched LoRA rank and compute.

The stacking baseline applies pedagogical DPO followed by cultural preference training using the CARE recipe, which reports up to a 29% gain in cultural scores. This tests whether two off-the-shelf fixes in sequence already match a joint, turn-deep objective.

We run three ablations to isolate the key ingredients. The first uses our full objective with pairs drawn only from turns 1 to 3, which isolates turn depth. The second removes neutral calibration, which tests the NormAd over-localization risk. The third uses only pedagogically drifting pairs or only culturally drifting pairs. If coupling is real, single-axis pairs should transfer partly to the other axis.

"Beating" a baseline means a smaller area under both decay curves across turns 6–12, with 95% confidence intervals from a dialogue-level bootstrap. It also means no worse neutral-context over-localization than CoCoA, and a general-benchmark drop within CoCoA's reported 0.38-point bound.

Why not just…?

Why not just re-insert the persona and pedagogy instructions every few turns? This is the most serious threat, and we have no published evidence against it. Periodic re-prompting directly counters instruction fade and costs nothing to train. We therefore include it as a baseline with re-injection every 2 and every 4 turns. If it matches our method on both curves, the honest conclusion is that late-turn drift is a context-attention problem and not a preference problem. The diagnostic and coupling result would then become the main contribution, and the training method would not be. One reason to expect it to fall short is the CAMeL finding: models prefer Western entities 40% to 65% of the time even when the prompt itself explicitly references Arab culture. Fresh cues alone may not be enough.

Why not just use a larger frontier model as the tutor? Having Beer after Prayer? found that larger multilingual models showed stronger Western bias than smaller monolingual ones. NormAd's neutral-context results show frontier models over-reading culture. We run GPT-4-class models through the same testbed as an unaligned upper reference. If a frontier model shows flat curves, the method's value narrows to open-weight deployment, and we will say so.

Why not just stack existing pedagogical and cultural alignment? Both component methods are trained on single-turn or early-turn data. That is exactly the setting where the Pedagogical Alignment paper shows imitation looks fine before decaying. The stacking baseline and the turn-depth ablation together settle this. If stacking matches us, turn depth is not the missing ingredient.

Proposed Experiments and Decisive Checks

The first decisive check is the coupling regression on base-model dialogues across three languages. We test whether $\beta_2 > 0$ after controlling for depth, and whether it replicates across languages and subjects. A null result there redirects the project toward separate fixes.

The second check is the full baseline comparison on held-out seeds written by different teachers. For each method we report per-turn $S_t$ and $W_t$ curves, neutral-context over-localization rates, CAMeL grounded and neutral scores for continuity with CoCoA, and general benchmarks.

The third check is cross-axis transfer. Training on culture-only drift pairs should reduce pedagogical drift, and vice versa, if the two failures share a cause.

Finally, a small study with native-speaking secondary students in one Arabic-speaking setting compares the base model, re-prompting, and our tutor on post-dialogue problem-solving and perceived appropriateness. It also records whether students preferred hints or direct help. We report this preference as exploratory evidence on the culture-pedagogy question, not as a training target.

No reported result was found on Lacuna for any existing method's per-turn cultural reversion in tutoring dialogue. The testbed's base-model curves will therefore be the first such reference numbers.