Grapheme-to-phoneme conversion (G2P) turns a written word into its pronunciation. It is a basic component of speech synthesis and recognition. For most of the world's languages, a G2P model has only a few hundred labeled words to learn from. Unlabeled written words are plentiful, though, and for many scripts a linguist can write a short rule that splits a word into syllables well enough to be useful. That rule does not produce a pronunciation. It only says which letters form the vowel core of each syllable and which consonants belong to its start or end. This note proposes a specific way to use that cheap, noisy structure. A pronunciation model and a syllable tagger share one encoder. The tagger is trained on a large pool of unlabeled words carrying rule-generated syllable tags. The two objectives take turns on a randomized schedule, so the pronunciation task never has to see the noisy tags.
The question matters because the obvious ways of adding syllable knowledge contradict each other in the literature. Inserting predicted syllable boundaries into the input helped a structured-SVM letter-to-phoneme pipeline on English (Automatic Syllabification with Structured SVMs for Letter-to-Phoneme Conversion, word accuracy 84.67% to 85.59% on CELEX). Morpheme boundaries gave a 9.2% word-error reduction on unseen roots (Enhancing Sequence-to-Sequence Text-to-Speech with Morphology). Yet heuristic syllable boundaries inserted into the input of a neural transducer failed to beat the baseline on the SIGMORPHON 2021 low-resource benchmark (Linguistic Knowledge in Multilingual Grapheme-to-Phoneme Conversion). Our hypothesis is that the failure comes from where the noise enters, not from the syllable signal itself. A wrong boundary in the input corrupts the very string the model must pronounce. A wrong tag used as an auxiliary target only nudges shared features, and its influence can be controlled.
Core Claim
Rule-generated syllable-position tags help low-resource G2P under three conditions: they are used as a prediction target on a separate, larger pool of unlabeled words; they are not inserted into the input; and their share of updates is controlled by a stochastic schedule. The same tags used as input annotations will not help, which would reproduce the negative result of Lo & Nicolai. The claim is testable on its own terms. If rule tags help as targets and hurt as inputs at matched noise levels, that explains a live contradiction. If they help as targets only on the labeled words, the extra unlabeled pool is irrelevant and the proposal shrinks to a known idea.
The closest prior work does something different in each case. Van Esch et al. (Predicting Pronunciations with Syllabification and Stress with Recurrent Neural Networks) predict phonemes, syllable marks and stress as one output string. That requires gold syllabified pronunciations for every training word, which low-resource lexicons do not have. Wang & Sim (Integrating conditional random fields and joint multi-gram model with syllabic features for grapheme-to-phone conversion) feed syllabic features from an external syllabifier into a rescoring stage, which is a cascade. Kang et al. (Multi-Task Learning for Front-End Text Processing in TTS) supply the method we reuse: they cycle through tasks that have disjoint datasets, one head per step. Their tasks do not include pronunciation or syllable structure, however, and their cycle is a fixed round-robin. Prabhu & Kann (Frustratingly Easy Multilingual Grapheme-to-Phoneme Conversion) show that a second objective (phoneme-to-grapheme) helps G2P. Their auxiliary data is the same labeled set reversed, so it adds no information beyond the lexicon. Our auxiliary data is new words plus a linguist's rule.
Research Program
The model is a small character-level Transformer encoder-decoder of the kind used for CMUDict (Transformer Based Grapheme-to-Phoneme Conversion, about 2M parameters). A linear tagging head is added on top of the encoder. For each grapheme, the head predicts one tag from a Break-ONC scheme, following Bartlett et al.: onset, nucleus or coda (the start, vowel core or end of a syllable), plus a flag for whether a syllable boundary follows.
Each language gets a rule syllabifier of the kind Lo & Nicolai used. It marks vowel letters or vowel-bearing characters as nuclei, gives intervocalic consonants to the following syllable up to a list of licit onsets, and leaves the rest as codas. Writing this rule should take an hour per language, not weeks. That cost is the point, since the high cost of full rule-based G2P is a known limitation of rule systems. The rule runs over an unlabeled word pool $U$. The pool is built from WikiPron orthographic entries with pronunciations discarded, and every word that appears in any official train, dev or test split is removed.
Training alternates. At step $t$ the trainer draws a task. With probability $p_t$ it takes a G2P batch from the labeled set $D$ and updates the encoder, decoder and embeddings with sequence cross-entropy. Otherwise it takes a batch from $U$ and updates the encoder and tagging head with tag cross-entropy. With a fixed $p_t = p$ this is the ALRIGHT scheme of Fernando et al. (Mitigating Forgetting in LLM Supervised Fine-Tuning and Preference Learning), transplanted from language-model alignment to sequence transduction.
We also test a primary-guarded variant of their MAXRIGHT rule. Let $\ell_{\text{g2p}}$ be the G2P loss on a held-out slice of the training words, and let $\ell^{*}_{\text{g2p}}$ be its best value so far. We set
$$p_t = \min\!\big(1,\; p_0 + \beta\,[\ell_{\text{g2p}}(t) - \ell^{*}_{\text{g2p}}]_+ \big),$$
where $p_0$ is the base probability of a G2P step, $\beta$ is a sensitivity constant, and $[\cdot]_+$ keeps only positive values. When the primary task is losing ground, auxiliary steps become rarer. The slice is carved out of $D$ so the official dev set stays untouched for model selection. This is a deliberate departure from symmetric MAXRIGHT. Here the auxiliary task is instrumental: we care about G2P accuracy alone, not about a Pareto front.
We are not relying on the memory argument for alternation. The 35% memory saving Fernando et al. report concerns billion-parameter models and is irrelevant for a 2M-parameter G2P network. Alternation is proposed for two other reasons. First, the two tasks draw from disjoint data of very different sizes: about 800 labeled words against tens of thousands of unlabeled ones. Second, the schedule is a direct handle on how much noisy supervision reaches the shared encoder. Whether this handle does better than a tuned static weight is an empirical question, and the experiments below are built to answer it.
The analysis component isolates the noise explanation. CELEX provides gold syllabification for English, German and Dutch. On CELEX we can therefore corrupt gold tags at controlled rates, and also substitute the rule tags. The same tags are then fed either as input annotations or as auxiliary targets. We also probe the encoder, measuring how well a frozen linear probe recovers gold syllable positions. This tells us whether the auxiliary task changes what the encoder represents, or merely regularizes it.
Baselines
The naive reference point is a single-task Transformer of identical size, trained only on the 800 labeled words per language. Tuning follows the same protocol as every other system: the same hyperparameter search budget of a fixed number of trials per language, selected on the official dev split, with five seeds reported as mean and spread. This baseline tells us whether any structure helps at all.
The strongest reported competitors on the SIGMORPHON 2021 low-resource tier are the systems studied by Lo & Nicolai. Their imitation-learning neural transducer reaches a macro-average WER of 25.1%. Their vowel-and-diacritic penalty system (VP) reaches 24.1% (Linguistic Knowledge in Multilingual Grapheme-to-Phoneme Conversion). VP is the strongest competitor on this question because it is also a cheap way of injecting rule-derived phonological knowledge. It is the bar to clear: beating it means a macro-average WER below 24.1% on the same test split, with the improvement larger than seed variance. The shared-task overview reports only a 4% relative WER reduction for the top low-resource system (Results of the Second SIGMORPHON Shared Task on Multilingual Grapheme-to-Phoneme Conversion). We will quote that result as given, since the overview's absolute figure is not in the file. We will run VP's penalty idea on our Transformer so that architecture is controlled. We also report the published transducer numbers so that the choice of architecture itself is visible.
A second strong competitor is Prabhu & Kann's G2P+P2G multi-task Transformer. On SIGMORPHON 2020 medium resource it reports 14.65 WER and 3.34 PER averaged over the dev set (CU-2), and multi-task training accounted for a 0.88 WER gain (Frustratingly Easy Multilingual Grapheme-to-Phoneme Conversion). We run it on both the 2021 low tier and the 2020 medium tier with our tuning budget. We also run it combined with the syllable auxiliary, because the two auxiliaries could be complementary.
Three further systems act as ablations of the key ingredients. The first is a replication of rule syllable boundaries as input tokens, which is the Lo & Nicolai negative result. It isolates target use from input use. The second is the syllable auxiliary trained only on the labeled words' orthography. It isolates the unlabeled pool. The third is a static joint objective: at each step one G2P batch and one unlabeled batch are drawn and $L_{\text{g2p}} + \lambda L_{\text{syl}}$ is minimized. Here $\lambda$ is tuned over the same number of trials as $p$ and $\beta$, and it is also paired with random loss weighting as in Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning. This isolates the schedule. We expect the syllable auxiliary to beat the input-token and labeled-only variants. For the schedule, success means the guarded variant beats the tuned static sum. If it does not, we will report that alternation is a convenience rather than a contribution.
Why not just…?
Why not just use a tuned weighted sum of the two losses? This is the most serious threat. At this model size the memory benefit of alternation disappears. In expectation, a stochastic schedule with constant $p$ applies roughly the same pressure as a static weight of about $(1-p)/p$. Only the adaptive, primary-guarded schedule makes a distinct prediction: it should reduce harm on languages whose rule tags are poor, such as Khmer and Adyghe, where a fixed $\lambda$ tuned on the macro average may overweight bad supervision. The per-language comparison against a tuned static sum and random loss weighting settles this. We commit in advance to reframing the contribution around the auxiliary-target idea if the schedule does not win.
Why not just run a rule-based G2P over the unlabeled words and train on the pseudo-pronunciations? This uses the same unlabeled pool more directly, and it is a genuine alternative. The difficulty is that full rule G2P is exactly the part of rule-based conversion that is expensive to write and brittle on loanwords and exceptions, as the rule-based G2P literature documents. Syllable rules are simpler and their errors are local. We will nonetheless build light rule-G2P pseudo-labels for the languages with near-transparent orthographies (Italian, Romanian, Welsh, Latvian) and compare directly. If pseudo-labeling wins there, the honest conclusion is that syllable auxiliaries matter only where full rules are too costly to write. We would report that boundary.
Why not just let a Transformer learn syllable structure implicitly? With enough data it likely does. Van Esch et al. and Bartlett et al. show that explicit structure still helps, but they worked with thousands of words. The probing experiment measures whether 800 words suffice to encode syllable position. The labeled-only auxiliary ablation tells us whether any gain comes from new words rather than from a redundant label.
Why not just ensemble? Prabhu & Kann found that ensembling gave more than multi-task training (1.50 against 0.88 WER), and that the two combine. Ensembling is orthogonal to our question. We report single models throughout, plus one five-model ensemble of the best configuration, to show whether the gain survives ensembling.
Proposed Experiments
The main experiment covers all ten SIGMORPHON 2021 low-resource languages. Metrics are word error rate and phoneme error rate on the official test split, compared against the naive baseline, VP and the P2G multi-task model. We then vary the size of the unlabeled pool from zero to the full WikiPron remainder, which tests whether gains grow with more rule-tagged words, as the hypothesis predicts. Transfer to more data is checked on the SIGMORPHON 2020 medium tier, where we expect smaller gains. Where it can be obtained, we will also check English CELEX, against the structured-SVM pipeline's 85.59% word accuracy.
The decisive check is the CELEX noise sweep. Gold syllable tags are corrupted at increasing rates and supplied either as input tokens or as auxiliary targets. The target curve should degrade gracefully, while the input curve should cross below the no-syllable baseline at a noise rate close to that of the rule syllabifier. If both curves behave the same, the explanation for Lo & Nicolai's result is wrong and the proposal fails at its core. A second check is per-language: gains should correlate with the rule syllabifier's accuracy, which can be estimated on a small hand-checked sample. They should be near zero where the rule is poor, and the guarded schedule should prevent those languages from falling below baseline.
References
- Automatic Syllabification with Structured SVMs for Letter-to-Phoneme Conversion
- Enhancing Sequence-to-Sequence Text-to-Speech with Morphology
- Linguistic Knowledge in Multilingual Grapheme-to-Phoneme Conversion
- Predicting Pronunciations with Syllabification and Stress with Recurrent Neural Networks
- Integrating conditional random fields and joint multi-gram model with syllabic features for grapheme-to-phone conversion
- Multi-Task Learning for Front-End Text Processing in TTS
- Frustratingly Easy Multilingual Grapheme-to-Phoneme Conversion
- Transformer Based Grapheme-to-Phoneme Conversion
- Mitigating Forgetting in LLM Supervised Fine-Tuning and Preference Learning
- Results of the Second SIGMORPHON Shared Task on Multilingual Grapheme-to-Phoneme Conversion
- Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning