Code-mixed sentiment datasets are text where writers switch between languages mid-sentence, often typing Hindi, Tamil or Bengali in the Roman alphabet with no fixed spelling. Classifiers trained on this text are weakest exactly where humans disagree. Hinglish annotators in SemEval-2020 agreed only 55% of the time, and the best Tamil system scores 0.26 F1 on "Mixed Feelings." A single gold label per post hides the fact that many posts support two readings. We propose to recover that structure offline. For each training post, two instances of one language model argue for competing sentiment readings. One reads the literal translated meaning; the other reads the post as colloquial slang, sarcasm or pun. Every claim must quote the exact words it relies on, and a judge strikes any claim the opponent refutes or that quotes nothing. What is left is a set of arguments that survived debate, each tied to input words, plus how strongly each label was defended. We then train a small, cheap classifier to match that label spread and to tag the words the surviving arguments cited.
In two sentences: debate is used only at training time, to turn each ambiguous code-mixed post into a spread over labels and a set of evidence words that survived cross-examination. A small multilingual encoder learns from both, so the deployed model pays no language-model inference cost but should handle the mixed and ambiguous classes better than one trained on hard gold labels.
Core Claim
Explanations help classification when they carry information the gold label lacks. On code-mixed sentiment, the missing information is how contested a post is and which words drive each side. Our claim is that span-grounded debate recovers this more faithfully than a single model's chain-of-thought or a vote over independent samples. The improvement should show up mainly on minority and ambiguous classes and in calibration, not in headline accuracy on the dominant Positive class. We state the claim so it can fail. If a vote over independently sampled rationales, matched in token budget and filtered by the same quote-checking rule, trains an equally good student, then the debate itself adds nothing. We would report that the useful ingredient is disagreement-aware soft labels and grounding, not dialectic.
Relation to Prior Work
Three papers motivate the proposal by showing the gap. Reasoning Beyond Labels had single LLMs explain sentiment in code-mixed Kenyan WhatsApp messages. Open-weight models broke down on sentiment-flipped counterfactuals and were overconfident, and Mistral-7B classified well while explaining worst. That paper only measures single-agent explanations. We use debate to produce training signal on Indic and Dravidian benchmarks. SemEval-2020 Task 9 documents the 55% agreement rate. byteSizedLLM@DravidianLangTech 2025 fine-tuned a transliteration-aware XLM-RoBERTa and still reached only 0.26 F1 on Mixed Feelings, so better pretraining alone does not fix the ambiguous classes.
Four papers supply the method. ERD is the closest precedent. It extracts the relevant segment, reasons over it, then runs a for-and-against debate with a judge for cognitive distortion classification, raising multi-class F1 from 15.28 to 24.27. ERD uses debate as the inference-time classifier on monolingual English, with clinical personas. We run debate offline, assign roles by reading perspective (literal vs. colloquial), and treat the transcript as supervision for a student. We reuse ERD's finding that gains saturate after two rebuttal rounds to cap our cost. Zero-shot Code-Mixed Offensive Span Identification showed that Tamil-English classifiers can be pushed toward token-level rationales. We borrow its idea of a word-tagging auxiliary target, but our tags come from debate-surviving quotes rather than LIME or Integrated Gradients. EGM-PM feeds single-LLM explanations to a downstream predictor and itself warns that such explanations can be "plausible but ungrounded" and degrade in low-resource settings. Our quote-and-strike rule is a direct response to that warning. Emergent Temporal Reasoning shows a small student can surpass the teacher whose rationales it learns from, which is why we expect the student to exceed the debating model.
Two papers constrain the design. Examining Inter-Consistency via Debate found that a weaker debater can drag a stronger one below its solo accuracy. We therefore debate one model against itself under different role prompts rather than pairing mismatched models. A Diagnostic Study of Multi-Agent LLMs for Real-World Debates showed that consensus is a poor signal of truth, while process quality is a good one. Their process quality measures whether agents engage with each other's arguments and whether one agent dominates. High-process debates recovered from an incorrect initial majority 74% of the time, versus 40% for low-process debates. We do not treat the judge's verdict as a label. We use their influence-asymmetry measure, which detects a single agent dominating the exchange, to down-weight lopsided debates. TMAK-Plus used cooperating critic agents for aspect sentiment but passed structured JSON messages rather than adversarial rebuttal, and it did not train a student.
Research Program
For each training post $x$ with gold label $y$, a single open-weight instruction model plays three roles. The literal reader argues for the sentiment implied by translating each language span at face value. The colloquial reader argues for the reading implied by slang, transliterated idiom, sarcasm or emoji. The judge scores the result. Each argument must quote a contiguous span of $x$ verbatim; a string-match check drops any claim whose quote is absent from the post.
Debate runs two rounds of rebuttal. In each round, an agent either refutes an opposing claim, quoting counter-evidence, or concedes it. The judge then outputs, for every sentiment class, how much unrefuted grounded support remains. We normalize these scores into a distribution $q(\cdot \mid x)$ over classes. We also compute the influence-asymmetry measure of the diagnostic study and use it to set a per-example weight $w(x)$ that shrinks toward the gold label when one role dominated.
The student is XLM-RoBERTa, or the transliteration-aware variant of byteSizedLLM, with two heads: a sentence classifier $p_\theta$ and a token tagger $t_\theta$. It minimizes
$$ \mathcal{L} = \mathrm{CE}(y, p_\theta) + \lambda\, w(x)\, \mathrm{KL}\big(q \,\|\, p_\theta\big) + \mu\, \mathrm{BCE}(s, t_\theta). $$
Here $\mathrm{CE}$ is cross-entropy against the gold label, and $\mathrm{KL}$ pulls the student toward the debate distribution $q$. $s$ is the binary mask of input tokens quoted by surviving arguments, $\mathrm{BCE}$ is the per-token loss for predicting that mask, and $\lambda$ and $\mu$ are tuned on the development set. At inference the student alone runs, so there is no generation latency. This addresses the inference-cost objection raised against GRACE and chain-of-thought distillation.
The analysis half asks whether debate outputs track real human ambiguity. On SentiMix, where agreement is low, we test whether the entropy of $q$ is higher on posts whose gold label is Neutral or that lie near the class boundary. On BnSentMix and the Dravidian sets we test whether $q$ puts mass on the Mixed class for gold-Mixed posts more often than a single model's confidence does.
Baselines
The do-nothing reference is the student encoder fine-tuned on gold labels alone with cross-entropy, at the same learning-rate and epoch budget. As a second naive point we include the character n-gram logistic regression of TueMix, which reports weighted F1 0.685 on Hinglish. It is included because on this data shallow models repeatedly match deep ones, so any gain must clear it.
The strongest existing methods differ by dataset, and we compare against their reported numbers while also re-running them. On DravidianLangTech 2025 Tamil, the strongest is byteSizedLLM's transliteration-aware XLM-RoBERTa with an Attention-BiLSTM. It reports macro F1 0.50, weighted F1 0.63 and Mixed Feelings F1 0.26. On SentiMix Hinglish, the task overview reports 75.0% F1 for KK2018's XLM-R with adversarial training. On BnSentMix, BnSentMix reports BERT at 72.7% accuracy and 71.7% F1. For FIRE 2020, Sentiment Analysis of Dravidian Code Mixed Data reports weighted F1 0.65 for Tamil and 0.68 for Malayalam. We re-implement the byteSizedLLM encoder as our student backbone so that any gain comes from the training signal, not the architecture.
Beating these means a statistically significant improvement in macro F1 across five seeds and at least 0.05 absolute on the minority class (Mixed Feelings or Mixed), without losing more than one point of weighted F1. We also require lower expected calibration error, which measures how far predicted confidence departs from observed accuracy.
The inference-time debate baseline is ERD-style debate applied directly to test posts with the same model. It tells us whether distillation loses anything relative to paying for debate at deployment.
The single-agent explanation baseline trains the student on one chain-of-thought rationale per post, following the EGM-PM recipe. The label distribution comes from the model's stated confidence, and the tags come from the same quote rule.
The key ablation is self-consistency with grounding. It samples $k$ independent quote-grounded rationales per post, with $k$ chosen so total generated tokens equal the debate's, and applies the same string-match filter. The vote share becomes $q$ and the union of cited words becomes $s$. This isolates rebuttal and concession as the only difference. Further ablations drop the KL term, drop the tagging term, replace role prompts with generic pro/con stances, and remove the influence-asymmetry weight. All students share one hyperparameter search budget on the development split.
Why not just…?
Why not just fine-tune XLM-RoBERTa on the gold labels? That is our do-nothing baseline, and the evidence says it plateaus on exactly the classes we target: 0.26 F1 on Mixed Feelings even with transliteration-aware pretraining in byteSizedLLM. Gold labels carry no information about which posts are contested. If our student fails to raise minority-class F1 over this baseline, the proposal fails.
Why not just use character n-gram logistic regression? It is fast and robust to spelling variation, and TueMix got within about seven points of the SentiMix winner with it. Its known weakness is non-literal meaning. Sentiment Analysis of Dravidian Code Mixed Data identifies sarcasm and inverted polarity as its main failure mode. We will report per-class results and a sarcasm-heavy slice of the test set; if n-grams tie us there, the debate signal is not doing its job.
Why not just have one model explain each post and train on that? This is EGM-PM transplanted to code-mixed text. Reasoning Beyond Labels found single open-weight models produce poorly calibrated, low-quality code-mixed explanations, which suggests the signal would be noisy. That is indirect evidence, so the single-agent explanation baseline tests it directly.
Why not just sample several rationales and vote? This is the most serious threat, because disagreement across samples already yields a soft label, and quote-filtering already enforces grounding. A Diagnostic Study of Multi-Agent LLMs for Real-World Debates argues that agreement among agents is a weak signal. Examining Inter-Consistency via Debate found debate beat simple collaboration baselines only modestly, 81.63% against about 79%. We therefore do not assume debate wins. The token-matched self-consistency ablation settles the question. If it matches the full method, we reframe the contribution around grounded soft labels and drop the debate claim.
Proposed Experiments and Decisive Checks
We run all methods on SentiMix Hinglish, BnSentMix, FIRE 2020 Tamil-English and Malayalam-English, and DravidianLangTech 2025 Tamil sentiment. We report macro F1, per-class F1, weighted F1 and calibration error, averaged over five seeds, with debate generation run once on training data and its cost reported in tokens.
The first decisive check compares the full method against token-matched grounded self-consistency on minority-class F1. This result determines whether debate is part of the story at all.
The second check tests faithfulness of the tags. We delete the student's top-tagged words and measure the drop in its predicted probability, against deleting random words of equal count, and against Integrated Gradients as used in Zero-shot Code-Mixed Offensive Span Identification. This shows whether the tags reflect what the classifier actually uses.
The third check asks whether debate entropy predicts human disagreement. We test this on SentiMix posts relabeled by three fresh bilingual annotators on a sample of 500. The no-debate student's predictive entropy serves as the control.
The fourth check tests robustness to rewording. We generate sentiment-flipped counterfactuals using the transformation types of Reasoning Beyond Labels (negation, tone shift, emoji change) and measure whether the student flips correctly more often than the gold-only student.
A clean positive result would show gains concentrated on Mixed and Neutral posts, beyond both self-consistency and single-agent rationales, with tags that pass the deletion test.
References
- Reasoning Beyond Labels: Measuring LLM Sentiment in Low-Resource, Culturally Nuanced Contexts
- SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets
- byteSizedLLM@DravidianLangTech 2025: Sentiment Analysis in Tamil Using Transliteration-Aware XLM-RoBERTa and Attention-BiLSTM
- ERD: A Framework for Improving LLM Reasoning for Cognitive Distortion Classification
- Zero-shot Code-Mixed Offensive Span Identification through Rationale Extraction
- Enhancing the Interpretability of NLI Models Using LLMs and Active Learning Algorithms
- Emergent Temporal Reasoning: Distilling Chain-of-Thought for Zero-Shot Combinatorial Generalization
- Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate
- A Diagnostic Study of Multi-Agent LLMs for Real-World Debates
- TMAK-Plus at SIGHAN-2024 dimABSA Task: Multi-Agent Collaboration for Transparent and Rational Sentiment Analysis
- GRACE: Generative Representation Learning via Contrastive Policy Optimization
- TueMix at SemEval-2020 Task 9: Logistic Regression with Linguistic Feature Set
- BnSentMix: A Diverse Bengali-English Code-Mixed Dataset for Sentiment Analysis
- Sentiment Analysis of Dravidian Code Mixed Data