A polysynthetic word, such as one in St. Lawrence Island Yupik, Chukchi or Nahuatl, can hold a whole clause: a verb root, an incorporated object, and a string of aspect, mood and agreement suffixes. Parsers for these languages increasingly treat each morpheme as its own node in a syntax graph, and this helps. In Yupik, morpheme-level parsing with gold segmentation reaches 91.82% unlabeled attachment score, against 88.86% at the word level (Expanding Universal Dependencies for Polysynthetic Languages: A Case of St. Lawrence Island Yupik). An encoder that lets morphemes exchange information inside a word, though, faces the same risk that graph neural networks (models that update each node by averaging its neighbors' features) face on graphs. Repeated averaging acts like a blur. It keeps what nodes share and erases what distinguishes them. This failure is called over-smoothing. Inside a polysynthetic word, what the morphemes share is a pile of frequent, repetitive suffixes. What distinguishes them is often a single rare root. That is exactly the information a parser needs, and exactly the information averaging destroys.
This note proposes to test that hypothesis directly and to fix it only if it holds. We would measure how "flat" the morpheme representations inside each word become, using the rank-based diagnostics that recent graph work showed track accuracy far better than older energy measures. We would check whether that flatness grows with the number of morphemes per word and whether it predicts attachment errors on word-internal arcs. We would then test whether adding a high-pass channel helps. That channel carries each morpheme's difference from its word-mates, and the model decides per morpheme how much to use it. The goal is to recover those arcs, especially for rare roots.
Core Claim
In morpheme-level encoders for polysynthetic languages, word-internal representation collapse increases with synthesis (morphemes per word). It is concentrated on low-frequency roots, and it accounts for a measurable share of word-internal parsing errors. A per-morpheme choice between a smoothing channel and a difference channel repairs it better than simply keeping the model shallow or adding residual connections.
The claim is falsifiable at three points, and in order. First, collapse may not grow with synthesis. Second, collapse may not predict errors. Third, the difference channel may help no more than a residual connection does. If any of these fails, the proposal reduces to a negative result about the transfer of over-smoothing to morphology, which is still worth reporting.
Why This Pairing, and What Is New
The graph side supplies the mechanism and the measuring tools. Complete the Missing Half splits the identity operator into a low-pass part (averaging) and a high-pass part (difference from neighbors). It shows that restoring the high-pass half improves GCN by roughly 10% on graphs where neighbors differ in kind. Geometric Scattering Attention Networks lets each node choose among frequency channels and lifts Chameleon accuracy from 28.2% (GCN) to 61.2%. When Do GNNs Work gates aggregation per node using neighborhood entropy and center–neighbor similarity, which is especially valuable when labels are scarce (73.3% vs 59.2% on Cora at a 1% label rate). Are we measuring oversmoothing in graph neural networks correctly? shows that effective rank, the entropy of the singular value spectrum, correlates with accuracy at an average Pearson 0.91, while Dirichlet energy does not. It also warns that high rank is necessary but not sufficient, since noise has maximal rank. That warning constrains us: rank alone cannot be the success criterion, so we pair it with a root-identity probe.
The polysynthesis side supplies the regime. The two treebank papers (Yupik, Highland Puebla Nahuatl) and the Chukchi work on noun incorporation, which represents incorporated nouns as separate enhanced-dependency nodes (Dependency annotation of noun incorporation), give us morpheme graphs. Crucially, the nodes are linked across roles, root to suffix and verb to incorporated object. This is the neighbors-differ setting in which averaging is known to hurt. Quantifying Synthesis and Fusion supplies a per-word synthesis measure to stratify by. Neural Polysynthetic Language Modelling is the closest neural treatment of morphological trees. It compresses finite-state parse trees into vectors with tensor product representations, which bind role vectors to filler vectors. It does not study message passing or representation collapse, and we contrast against it below.
None of these papers measures collapse inside polysynthetic encoders. None of the graph papers has applied its filters to morphology. The contribution is the causal chain from synthesis to collapse to errors to repair, not a new filter.
Research Program
We build a morpheme-level biaffine dependency parser, meaning one that scores every head–dependent pair with a bilinear function, on three encoders that share input embeddings (morpheme form plus character CNN). The first is a sequence BiLSTM of the kind standard parsers use. The second is a hierarchical message-passing encoder with $L$ layers over a graph linking adjacent morphemes and linking every morpheme to a word hub node. The hub makes each word a star, so one round trip through it replaces every morpheme with roughly the word mean. That is the strongest version of the smoothing we hypothesize. The third is the same message-passing encoder with a two-channel update:
$$h_u^{l+1} = h_u^l + (1-g_u)\,W_{lo}\,\bar h_{N(u)}^l + g_u\,W_{hi}\,(h_u^l - \bar h_{N(u)}^l),$$
Here $h_u^l$ is morpheme $u$'s vector at layer $l$, $\bar h_{N(u)}^l$ is the mean over its neighbors (including the hub), $W_{lo}$ and $W_{hi}$ are learned matrices, and $g_u\in[0,1]$ is a per-morpheme gate. The gate is computed from center–neighbor cosine similarity and the entropy of the neighbors' predicted morpheme classes, following the ALa signals of When Do GNNs Work. The residual term is kept deliberately, so that any gain must come from the difference channel rather than from identity preservation alone.
For diagnostics, we compute for each word $w$ with $n_w$ morphemes the normalized effective rank of its morpheme matrix at each layer. This is $\mathrm{Erank}(H_w)/\min(n_w,d)$, where $d$ is the hidden size. We normalize because a word of $n$ morphemes cannot have rank above $n$. We also train a linear probe that recovers the root lemma from the root morpheme's final vector, and we report its accuracy stratified by training-set root frequency, with hapax roots as a separate bin. We then regress word-internal arc correctness on synthesis, normalized rank and root frequency. This lets us ask whether rank carries predictive signal beyond synthesis itself.
Baselines
The naive reference point is the word-level parser, whose reported Yupik score is 88.86% UAS (Park et al.). It shows how much morpheme-level structure is worth at all.
The strongest existing result is the morpheme-level Yupik parser with gold segmentation, at 91.82% UAS in the same paper. For Nahuatl it is the neural pipeline of Pugh & Tyers, at 79.8% UAS and 72.7% LAS in-domain. For Chukchi enhanced-dependency parsing there is no reported result found on Lacuna, so we report our own sequence-BiLSTM parser there. These are the right competitors because they are the published numbers for exactly the graphs we use. Our reimplemented BiLSTM parser must first match them within about half a point on the same splits before any comparison counts.
All encoders get the same embeddings, the same hidden size, a matched parameter count within 5%, and an equal hyperparameter search of 30 trials over learning rate, dropout and depth. Every configuration is run with five seeds. Beating the baseline means a word-internal-arc UAS gain larger than the seed standard deviation on at least two of three languages, concentrated in the high-synthesis and rare-root bins.
Four ablations isolate the key ingredient. The first uses the same encoder with $g_u$ fixed at 0, which gives residual smoothing only. The second fixes $g_u$ at 0.5, a fixed filterbank as in Complete the Missing Half. The third removes the hub node, which shows whether the star structure drives the collapse. The fourth replaces our gate with frequency-channel attention as in GSAN, restricted to one- and two-hop channels to keep cost comparable.
Why not just…?
Why not just keep the encoder shallow and add residual connections? Morpheme graphs have small diameter, so this is the most serious alternative. The ablation with $g_u$ fixed at 0 and $L\in\{1,2\}$ is exactly this model. If it matches the gated model in the high-synthesis bin, we will report that no special filter is needed. Our reason to expect otherwise is structural. With a hub node, a single layer already averages the whole word, and averaging is known to hurt immediately in neighbors-differ settings (Complete the Missing Half), not only at depth. Rank collapse has also been observed by about ten layers even with normalization (Zhang et al.).
Why not just rely on character features or a pretrained multilingual encoder, which see the root's spelling directly? This is a real threat, because character information can re-inject root identity after smoothing. We therefore include character CNNs in every model. As a second test we add a frozen pretrained multilingual encoder's features to the BiLSTM parser. If the difference-channel gain disappears once these are present, the collapse is real but practically harmless, and we will say so. The evidence that pretrained models handle this morphology poorly is indirect: NLLB and LLMs struggle with Nahuatl morphosyntactic manipulation (AmericasNLP 2025), and language models underperform on morphologically complex languages (Why do language models perform worse…).
Why not just encode the finite-state morphological tree with tensor product representations? Neural Polysynthetic Language Modelling does this. Because it binds each morpheme to a role instead of averaging, it cannot over-smooth by construction. We add it as a word encoder feeding the same parser. It is a fair test of whether avoiding message passing entirely is better. The file reports no parsing result for it. It also yields one vector per word, so it cannot directly score word-internal arcs, which limits it to the word-level comparison.
Proposed Experiments
The first decisive check is correlational and comes before any method claim. On Yupik, Nahuatl and Chukchi, we test whether normalized within-word effective rank in the plain message-passing and BiLSTM encoders declines monotonically with synthesis. We then test whether it predicts word-internal arc errors after controlling for synthesis and root frequency. A null result here ends the method arm.
The second check is the intervention. It asks whether the gated difference channel beats both the residual-only and the fixed-filterbank ablations on word-internal UAS and LAS in the top synthesis tercile and on hapax-root probe accuracy, while matching them on low-synthesis words. We also check that rank rises together with probe accuracy, not rank alone, to rule out the "noise has high rank" failure. To test robustness to realistic segmentation errors, which Park et al. found degrade parsing, we repeat all runs with automatic segmentation.
Finally, the learned gates $g_u$ should be larger on roots and incorporated nouns than on inflectional suffixes. This would be linguistic evidence that the model uses the difference channel where the theory says identity matters.
References
- Expanding Universal Dependencies for Polysynthetic Languages: A Case of St. Lawrence Island Yupik
- Complete the Missing Half: Augmenting Aggregation Filtering with Diversification for Graph Convolutional Networks
- Geometric Scattering Attention Networks
- When Do GNNs Work: Understanding and Improving Neighborhood Aggregation
- Are we measuring oversmoothing in graph neural networks correctly?
- A Universal Dependencies Treebank for Highland Puebla Nahuatl
- Dependency annotation of noun incorporation in polysynthetic languages
- Quantifying Synthesis and Fusion and their Impact on Machine Translation
- Neural Polysynthetic Language Modelling
- Findings of the AmericasNLP 2025 Shared Tasks on Machine Translation, Creation of Educational Material, and Translation Metrics for Indigenous Languages of the Americas
- Why do language models perform worse for morphologically complex languages?