Mandarin text-to-speech has to decide how to pronounce characters that have several readings. 行, for example, is xíng in 银行以外 but háng in 银行. Large pretrained Transformers now get about 99% of these decisions right on the standard benchmark. A half-million-parameter bidirectional LSTM (a recurrent network that reads the sentence in both directions) gets 97.31% and fits on a phone. Most of the remaining errors fall on rare readings, where models fall back on the common pronunciation. We propose to keep the small recurrent encoder and change how it decides. For each polyphonic character, we store a compact set of the encoder's hidden states from training. We link them into a nearest-neighbour graph, spread the reading labels along that graph, and classify a new occurrence by where it attaches to the graph. The decision then follows the shape of the data cloud, measured by paths through nearby states, rather than straight-line distance, which can cut across empty space.
The bet is narrow and can be tested. Within one character, the hidden states for a rare reading may form thin, curved branches lying close to the majority reading's mass. A linear softmax, or a plain nearest-neighbour vote, can then fold those branches into the majority. Distances measured along the graph should not. If that is true, the gain should be concentrated on tail readings. It should also be predictable, character by character, from how much the graph distances differ from straight-line distances. If it is false, a plain Euclidean memory should do just as well, and we will know the geometry adds nothing.
Core Claim
A per-character label-diffusion memory, built on the hidden states of a small BiLSTM trained in the usual way, should improve accuracy on rare readings over three alternatives: the same encoder with a linear head, the same encoder with a Euclidean k-nearest-neighbour head, and the same encoder with class-balanced classifier retraining. The size of the improvement for each character should track a measurable "short-circuit index", the median ratio of graph distance to Euclidean distance among that character's stored states. The memory adds little inference cost. Each character holds at most a few hundred landmark states, and the diffused labels are precomputed. At test time the cost is one small nearest-neighbour lookup plus a weighted sum.
Relation to Prior Work
Recurrent models for this task supply the encoder we reuse and the comparison point. g2pM released the CPP benchmark and a 0.5M-parameter BiLSTM. Dai et al. found that a BLSTM head on frozen BERT beat Transformer and fully-connected heads. DLM combines a BiLSTM with linear attention and handles the long tail by decoupling two stages: it learns representations under natural sampling, then retrains the classifier under reverse sampling. All three end in a parametric linear classifier. Our proposal is closest to DLM's second stage, and it differs in one way. Where DLM retrains a linear layer on reweighted data, we replace that layer with a nonparametric diffusion over the training states.
The work that supplies the method comes from manifold learning. Tenenbaum estimates long-range geodesic distance (distance along the surface of the data rather than through the space around it) by chaining short Euclidean hops on a nearest-neighbour graph. We use this principle in "locally flat, globally curved" form. TPN learns per-example graph bandwidths and propagates labels in closed form, $(I-\alpha S)^{-1}Y$, beating Euclidean few-shot classifiers. We borrow both the bandwidth trick and the closed form. TPN, however, works transductively on vision episodes with a CNN backbone. We apply it inductively, as a precomputed per-character memory over recurrent states. GeoMM replaces cosine similarity with hierarchical graph geodesics so that it can separate hard negatives that share lexical items, and it reports only modest training overhead. We reuse its idea of clustering states into landmarks so the graph stays small.
Two papers motivate the work and also constrain it. Pellegrino and Chadwick show that RNN hidden states lie on low-dimensional manifolds that the network warps, stretching task-relevant directions and collapsing irrelevant ones. That supports curved per-class structure. It also implies a counter-hypothesis: the network may already flatten the classes enough for a linear head to suffice. Our short-circuit index is designed to tell these two apart. Cluster Trees on Manifolds shows that high curvature forces small connection radii, and that noisy or sparse samples make graph construction brittle. For that reason we treat neighbourhood size as a quantity to sweep and report, not one to quietly tune.
Research Program
The encoder is a g2pM-style BiLSTM trained on CPP or CVTE-Poly with masked cross-entropy over each character's candidate readings. A distilled variant is trained against g2pW's soft predictions. For each polyphone $c$, we collect the encoder states at every training occurrence of $c$. Readings with fewer than a few hundred occurrences are stored exactly. Majority readings are compressed by K-means into landmarks, with each landmark carrying its occurrence count. On these nodes we build a k-nearest-neighbour graph with TPN-style adaptive bandwidths. We form the symmetrically normalised affinity $S_c$ and precompute diffused label scores
$$F_c = (I - \alpha S_c)^{-1} Y_c D_c^{-1},$$
where $Y_c$ is the one-hot reading matrix of the nodes, $D_c$ is the diagonal matrix of per-reading total counts, and $\alpha \in (0,1)$ controls how far labels spread. The division by $D_c$ stops large readings from flooding the graph, which is our imbalance correction. A test state $h$ attaches to its $k$ nearest nodes with Gaussian weights $w_j$. It is assigned the reading maximising $\sum_j w_j F_{c,j}$. This score can optionally be interpolated with the linear head's logits under a single weight $\lambda$, tuned on the development set. The matrix inversion happens offline, once per character, on graphs of at most a few hundred nodes, so it avoids the $O(N^3)$ cost on the full corpus.
Alongside the head, we compute an analysis quantity for each character: the short-circuit index, defined as the median ratio of graph shortest-path distance to Euclidean distance between pairs of states that share a reading. We also estimate the local intrinsic dimension of each character's state cloud, which connects the measurements to Pellegrino and Chadwick's dimension bound.
Baselines
The naive reference point is the majority-reading baseline. It reaches 92.08% test accuracy on CPP (g2pM), and any method should be judged mainly on the 8% it gets wrong. The direct recurrent baseline is the g2pM BiLSTM with its linear softmax, which reports 97.31% on CPP. We retrain it ourselves under the same tuning budget as every other variant.
The strongest existing method is g2pW. It is a BERT encoder with a conditional weighted softmax that is guided by jointly predicted part-of-speech tags, and it reports 99.08% CPP accuracy. Close behind are DLM at 99.07% and BERT-MFA from Li et al. at 99.06%. We do not expect a 0.5M-parameter model to beat g2pW overall. Success means closing a substantial fraction of the gap between 97.31% and 99.08% at roughly 1/200 of the parameters. It also means matching or beating g2pW on averaged accuracy over tail readings. On CVTE-Poly, the ERNIE-base system with tokenization and part-of-speech features from CVTE-Poly is the reference. Its paper reports 0.9891 on refined CPP, but we found no reported CVTE-Poly table numbers on Lacuna, so we rerun it.
The ablations that isolate the key ingredient all share the same frozen encoder, the same landmark set, and the same development-set tuning grid. The first is a Euclidean k-nearest-neighbour head over the identical nodes, with count normalisation, and differs from our head only in having no diffusion. The second is DLM-style reverse-sampling classifier retraining. The third is logit adjustment using the reading priors. The fourth is a two-layer MLP head of matched size. The proposal counts as working only if the diffusion head beats every one of these on tail-reading accuracy, with bootstrap confidence intervals that exclude zero.
Why not just…?
Why not just distill a BERT teacher into the BiLSTM? This is the most serious threat to the efficiency argument. Distillation could close much of the head-class gap without any geometry. We therefore run the distilled BiLSTM as a baseline and also use it as an encoder for the memory. If the diffusion head adds no tail-reading gain on top of the distilled encoder, we will report that the geometric component is redundant once the teacher's knowledge is transferred.
Why not just use a Euclidean nearest-neighbour memory or a kNN-LM-style interpolation? If the states are already flattened, which is what Pellegrino and Chadwick's warping result predicts is possible, plain Euclidean neighbours will match diffusion. The matched Euclidean head settles this directly. The short-circuit analysis also predicts where any difference should appear. If gains do not correlate with the index across characters, we drop the geodesic claim.
Why not just rebalance the loss or retrain the classifier as DLM does? DLM's reverse sampling reaches 99.07% with a large encoder. Its own stated limitation is RoBERTa latency, which says nothing about whether it fixes the tail with a small encoder. Running DLM-style retraining and logit adjustment on our exact encoder separates "imbalance correction" from "geometry". This is why our diffusion also divides by reading counts.
Why not just add a part-of-speech tagger? Pipelines inherit segmentation errors. Xu et al. reached 93.82% polyphone accuracy despite large gains from part-of-speech tags, and in Dai et al. an LSTM with part-of-speech embeddings scored 93.1% against 95.9% end to end. Part-of-speech features are also orthogonal to our head, so we test them as an added input.
Proposed Experiments
We evaluate on CPP and on the refined CPP test set. Because CVTE-Poly documents more than 70 labeling errors in CPP, differences of a few tenths of a percent near 99% are treated as noise. The primary benchmark is CVTE-Poly with its accuracy averaged by polyphone and pronunciation, which rewards tail readings. If the data are obtainable, we add the proper-noun and multi-polyphone test sets of Polyphone BERT, where a BERT classifier scores 91.9% and 83.0% and Polyphone BERT scores 93.0% and 87.2%. We report overall accuracy, macro accuracy over readings with under 1% within-character frequency, parameters, memory size, and CPU latency per sentence.
We run three decisive checks. The first is the head-to-head against the Euclidean memory on identical nodes. The second is a regression of per-character gain on the short-circuit index: a significant positive slope supports the geometric story, and a flat slope refutes it. The third is a sweep over neighbourhood size $k$ and landmark count, showing whether gains survive away from a single tuned setting, which the curvature results in Cluster Trees on Manifolds give reason to doubt. A negative outcome on the first two checks would still be informative. It would show that small recurrent encoders already flatten per-character reading geometry, and that the remaining gap to Transformers is a matter of knowledge rather than of the metric.
References
- g2pM: A Neural Grapheme-to-Phoneme Conversion Package for Mandarin Chinese Based on a New Open Benchmark Dataset
- Disambiguation of Chinese Polyphones in an End-to-End Framework with Semantic Features Extracted by Pre-Trained BERT
- DLM: A Decoupled Learning Model for Long-tailed Polyphone Disambiguation in Mandarin
- Mapping a Manifold of Perceptual Observations
- Learning to Propagate Labels: Transductive Propagation Network for Few-Shot Learning
- GeoMM: On Geodesic Perspective for Multi-modal Learning
- RNNs perform task computations by dynamically warping neural representations
- Cluster Trees on Manifolds
- g2pW: A Conditional Weighted Softmax BERT for Polyphone Disambiguation in Mandarin
- Improving Polyphone Disambiguation for Mandarin Chinese by Combining Mix-Pooling Strategy and Window-Based Attention
- CVTE-Poly: A New Benchmark for Chinese Polyphone Disambiguation
- Grapheme-to-phoneme conversion for Chinese text-to-speech
- A polyphone BERT for Polyphone Disambiguation in Mandarin Chinese