Editing the Decoder: Hierarchical Review to Stop Over-Reading Phonetically Cloaked Text

People hide offensive or censored words by writing them with characters that sound the same but mean something else. A Chinese commenter writes "spicy chicken" (là jī) for "rubbish" (lā jī). A Korean commenter swaps one sound unit inside a Hangul syllable. A writer spells a Korean word in Chinese characters chosen only for their Korean pronunciation. Language-model moderators lose a lot of accuracy on this kind of text. A second problem gets much less attention: once a system is allowed to read text "for its sound," it can find hidden messages almost anywhere. The philological literature on the Thousand Character Classic (Tcheonzamun) shows this risk clearly. Scholars reread the classic through Sino-Korean pronunciation and recover intimate domestic dialogues from a didactic text, for example a wife's anxiety for her husband (Kim et al.) or a joke about a husband looking like a radish top (Kim et al.). The file itself notes that this kind of decoding has no constraint grammar and invites confirmation bias. A moderation system that decodes this freely will flag innocent posts.

The claim of this proposal is that phonetic decoding and editorial skepticism should be split into different roles. A deterministic decoder proposes explicit restored readings of a post. A reviewer model judges each reading against the moderation policy. A separate critic, modeled on a handling editor, must cite concrete evidence that the literal reading is implausible before any decoded reading may count. We predict two things. First, phonetic information helps language models when it arrives as candidate rewrites rather than raw romanization, which would explain the "Pinyin Paradox." Second, the critic tier is what keeps false positives on benign text near the no-decoding baseline. Without the critic, decoding should raise recall on cloaked text and also raise false positives on clean text.

What Exists and What This Changes

Three lines of work motivate the proposal.

  • Attack work. Hiruncharoenvate et al. sample random homophones on Sina Weibo. They show that blacklisting the resulting variant space would collaterally block roughly 20% of daily traffic. Wang et al. build an 11-bit sound-and-shape code for adversarial character substitution, which transfers to GPT-4 with a 20% success rate. Both show the attack surface is large, and neither proposes a review process.
  • Diagnostic work. ToxiCloakCN measures the damage and reports that adding Pinyin to the prompt lowers accuracy.
  • Defense work. C²TU matches posts against a homophone graph and a toxic lexicon, then filters candidates with a single language model's likelihood score. Its authors list cross-lingual homophones and character splitting as out of scope. MESH fuses phoneme sequences into a Korean transformer through cross-attention and gains a few F1 points under attack. X-Guard routes multilingual input through a translation model before a safety judge, so a cloak that translation misses passes straight through.

None of these defenses asks how often decoding invents toxicity in clean text, and none separates "propose a reading" from "justify the reading." This proposal reuses C²TU's candidate-generation idea and PHISH's phonetic distances. It contributes the separation of those roles and the measurement of over-decoding.

The organizational design comes from two sources:

  • The review protocol. Adversarial Review shows on code benchmarks that a main-agent / reviewer / critic hierarchy with constrained critic outputs beats a flat 5-agent panel. The critic must answer AGREE, DISAGREE_EVIDENCE, or DISAGREE_CONCERN. We reuse this protocol. We change the domain, and we aim the critic at one specific failure, unwarranted decoding, rather than generic quality.
  • The incentive argument. Principal-agent analyses of delegated oversight (Dong) argue that front-line agents drift toward local incentives unless an intermediate tier audits them. The decoder plays that role here: it is incentivized to find something. Policy-as-Prompt supplies the practice of writing the moderation policy as a fixed rubric for the reviewer.

Research Program

The system has three tiers.

Tier 1: the decoder. It is deterministic and has no learned parameters. For Chinese it uses a Mandarin homophone graph with dialect shifts such as n↔l, following C²TU. For Korean it uses jamo-level substitutions ranked by phonetic (IPA) distance, following PHISH. For cross-script cases it uses a Sino-Korean reading table that maps runs of Chinese characters to Hangul syllables. For each post, the decoder outputs the literal text plus the top-k restored rewrites, each tagged with the substitutions that produced it.

Tier 2: the reviewer. An instruction-tuned language model scores each reading against the policy rubric.

Tier 3: the critic. A second instance of the model sees the literal post, the decoded reading the reviewer found most offensive, and the substitution trace. It must return one of three labels:

  • AGREE, meaning the decode is warranted.
  • DISAGREE_EVIDENCE, citing why the literal reading is coherent on its own.
  • DISAGREE_CONCERN.

Only a decode the critic endorses can raise the final label above the literal reading's score. Following Adversarial Review, the inner loop is capped at two rounds.

The analytical question is where the gains come from: the explicit rewrites, the separate critic, or the evidence constraint. We isolate each by removing it while holding the token budget fixed.

Baselines

The naive reference point. The same instruction-tuned model classifies the literal text zero-shot. The published anchor is GPT-4o zero-shot on ToxiCloakCN as reported by ToxiCloakCN: macro F1 0.764 on clean text, 0.677 on homophone cloaks, and 0.610 on emoji cloaks. We rerun this with our base model rather than relying on the published figure. We also run the Pinyin-augmented prompt from the same paper, because our second prediction is that it hurts while candidate rewrites help.

The strongest existing method on Chinese. C²TU restores the text and the restored text is then classified by the same reviewer model. On ToxiCloakCN its correction F1 is 70.01%, against 41.28% for the Simple-CSC spelling corrector (Breaking the Cloak!). That number measures word restoration, not final moderation accuracy. We found no reported result on Lacuna for downstream detection F1 of C²TU on ToxiCloakCN, so we produce that number ourselves. We count our system as beating C²TU if it matches C²TU's macro F1 on the cloaked split and shows a statistically lower false-positive rate on clean benign posts. Higher recall alone does not count.

The strongest existing method on Korean. seq-MESH fine-tuned on KOLD. Its reported advantage is 3.3% higher F1 than base KcBERT under a 10% single-jamo PHISH attack (PHISH in MESH). The undefended degradation is F1 drops of up to 33.5% for RoBERTa at 30% dual-jamo. We reimplement MESH on the same training split. Beating it means higher F1 at the 20% and 30% intensities without access to attacked training data.

Ablations, all compute-matched. Adversarial Review spends roughly 4.5 times the tokens of a single agent, so every baseline also gets self-consistency sampling up to the same budget. Hyperparameters, including k, get the same search on a shared development split. The three ablations are:

  • Decoder plus flat reviewer: the reviewer takes the maximum score over all candidates, with no critic.
  • Single-model self-critique: one model both judges and critiques in one prompt.
  • Free-form critic: the critic has no three-way output constraint.

Why not just…?

Why not just run C²TU and classify the restored text? This is close to what we build, minus the critic. C²TU picks a restoration by sentence likelihood. A cloaked toxic phrase and a coincidental homophone in a benign post can both score as likely, and C²TU has no step that asks whether the literal reading already made sense. If the decoder-plus-flat-reviewer ablation matches the full system on false positives, our critic claim fails. The benign probe below is built to detect exactly that.

Why not just give one strong model the list of candidate decodings? This is the most serious threat. The Pinyin Paradox may be about format, not about roles: raw romanization is noise, while rewritten sentences are ordinary text the model reads well. If the single-model self-critique ablation matches the three-tier system at equal tokens, the honest conclusion is that explicit rewrites are the contribution and the hierarchy is overhead. We would report it that way. Adversarial Review's gain over its single-agent baseline in code is a few points (75.2% vs 71.6% on SWE-bench Verified), so we expect a modest effect, concentrated on false positives.

Why not just fine-tune on augmented or phonetically fused inputs? MESH shows phonetic fusion helps, but each language needs retraining and attacked data. It also offers no account of the cross-script case, where Chinese characters spell Korean sounds. We compare against MESH on KOLD directly. On the cross-script set we predict MESH gains little, because its phoneme channel encodes Korean jamo rather than Sino-Korean readings of Chinese characters.

Why not just prompt a frontier model zero-shot? The ToxiCloakCN drops listed above, and the 20% GPT-4 transfer rate from Wang et al., indicate that scale alone does not close the gap. The zero-shot baseline stays in every table regardless.

Proposed Experiments

Chinese homophones. ToxiCloakCN homophone split, reporting macro F1. The emoji split falls outside the decoder's coverage. We report it with model-proposed candidates as a stress test rather than a claim.

Korean jamo attacks. KOLD Level A (KOLD, RoBERTa-large clean F1 78.8) and K-HATERS (K-HATERS) under PHISH at 10%, 20%, and 30% intensity.

Cross-script rebuses. No benchmark exists, so we build a new evaluation set. Offensive and benign KOLD comments are rewritten by replacing Korean words with Chinese characters chosen for their Sino-Korean reading, and native speakers verify that each rewrite stays readable.

Over-decoding probe (decisive check). The probe measures how often each system flags clean, human-verified non-offensive posts once decoding is switched on. It uses the non-offensive halves of the clean ToxiCN-derived data and KOLD. It also includes Tcheonzamun verses as a qualitative case: every one has a published "hidden" vernacular reading but carries no policy-violating content. The prediction is falsifiable: the flat-reviewer ablation's false-positive rate rises clearly above zero-shot, and the critic tier brings it back within a point.

Format test. Raw Pinyin or romanization in the prompt is compared against the same information given as rewritten candidates. If rewrites also fail to help, the rewrite-format half of the claim is wrong.