Graph models of conversation add a node for each speaker and link it to every utterance that person says. This lets a model remember who said what across a long exchange. We argue that the same design quietly erases what makes each utterance distinct. A speaker node averages all of its speaker's turns and sends that average back to each turn. Each pass pulls a person's angry outburst toward their typical calm demeanor. This is over-smoothing, the tendency of graph neural networks to make node representations converge as they repeatedly average over neighbors. It is usually treated as a problem of deep networks. In dialogue graphs it should appear even at two layers, because a speaker node is a hub: in two hops it connects every utterance its speaker produced, and in some models that spans many conversations.
The proposal has two parts. First, we diagnose the problem by measuring the effective dimensionality of utterance representations within each speaker, using rank-based tools that recent work shows track collapse better than classical energy measures. Second, we propose a fix that cannot be rewritten as a plain skip connection. Each utterance keeps two separate streams: a persona stream that passes through speaker nodes, and a deviation stream that never passes through them and moves only along reply, context and addressee edges. The prediction is concrete. If speaker hubs cause collapse, the gains should concentrate on utterances where a speaker's own emotion changes from their previous turn. They should not appear on changes between speakers, and they should not appear if we simply weaken the speaker edges.
Core Claim
The claim is that in heterogeneous dialogue graphs, the failure is driven by hubs rather than by depth. Speaker nodes collapse the within-speaker variation of utterance representations at the shallow depths dialogue models already use.
Keeping persona information and turn-specific deviation in separate streams preserves that variation. This should improve recognition of within-speaker emotion shifts. A degree-reweighted speaker edge, a residual connection, or a relation-compatibility energy model should not match it.
Background and Gap
Three dialogue graph models motivate the setting. ConGCN treats emotion recognition as classifying utterance nodes in a graph that also contains global speaker nodes, which link utterances across conversations. Its speaker edges are already weighted by inverse speaking frequency to curb dominant-speaker bias. That is evidence the authors felt pressure from hub nodes, but it is a degree fix, not a fix for loss of variation.
HeterMPC uses utterance and interlocutor nodes with six typed relations (reply, speak and address, each with an inverse) for multi-party response generation. DialogueGCN builds a windowed graph among utterances only, with speaker-dependent edges. All three use shallow message passing. None measures whether speaker routing homogenizes utterances.
Four over-smoothing papers supply the method and its constraints.
- Filterbank channels. Complete the Missing Half shows that standard aggregation is a low-pass filter. It adds a high-pass "diversification" channel $(I-\hat A)X$, which gains roughly 10% over GCN on the heterophilic Texas and Cornell graphs. It is defined only for single-relation graphs.
- Relation-aware energy. HALO derives heterogeneous GNN layers from gradient steps on an energy with relation-specific compatibility matrices. It reaches 76.20% on IMDB against 67.36% for Simple-HGN. It is evaluated only on static graphs.
- Spectral dialogue model. GS-MCC is the closest dialogue work. It splits a windowed multimodal utterance graph into low- and high-frequency parts with Fourier graph operators. However, it represents speakers as additive embeddings rather than nodes, so the hub mechanism we study does not exist in its graph.
- Measuring collapse. Are we measuring oversmoothing correctly? shows that Dirichlet energy misses collapse in trained networks. Effective rank, the exponentiated entropy of the normalized singular values, correlates with accuracy at an average Pearson r of 0.91. It also shows that residuals and normalization often fail to stop rank collapse. That last finding constrains our design: a fix that reduces to a residual is unlikely to help.
Research Program
Diagnosis. We take trained ConGCN-style and HeterMPC-style graphs on MELD, IEMOCAP and Ubuntu IRC. For each layer we compute two effective-rank profiles:
- across all utterances, the standard global measure;
- within each speaker, the effective rank of the matrix of that speaker's utterance states, averaged over speakers weighted by utterance count.
We compare three graph variants:
- with speaker nodes;
- with speaker nodes replaced by additive speaker embeddings, as in GS-MCC;
- with speaker nodes scoped to one dialogue rather than global.
The hub hypothesis predicts that within-speaker rank falls sharply after the first speaker hop, while global rank stays healthy. It also predicts the drop is largest with global speaker nodes. We correlate within-speaker rank with accuracy on within-speaker shift utterances. Rank is a necessary but not sufficient signal, so we treat it as a diagnostic, not a target.
Fix. Each utterance $i$ carries two states at layer $l$: a persona state $p_i^{(l)}$ and a deviation state $d_i^{(l)}$. Speaker node $s$ aggregates only persona states of its utterances. To avoid leakage in generation and online settings, it uses only turns preceding $i$. Utterances read persona back from their speaker. The deviation state is updated only over non-speaker relations:
$$d_i^{(l+1)} = \sigma\Big(\mathrm{LN}\big(d_i^{(l)} - \bar p_{s(i)}^{(l)}\big) W_0 + \sum_{r \in \{\text{reply},\text{context},\text{address}\}} \sum_{j \in N_r(i)} a_{ij}^{r}\, d_j^{(l)} W_r\Big)$$
Here $\bar p_{s(i)}$ is the speaker's persona summary, LN is layer normalization, $N_r(i)$ are $i$'s neighbors under relation $r$, $a^r_{ij}$ are attention weights, and $W$ are learned matrices. The subtraction is the speaker-relation instance of the filterbank high-pass channel.
By itself, a subtraction in one linear layer could be absorbed into a residual plus a speaker-mean term. The reason the design is not reducible is the routing: deviations travel along reply edges and never through the hub. A mild decorrelation penalty between $p$ and $d$ keeps the streams from merging. The classifier or decoder reads both streams.
Where it applies. On Ubuntu IRC, addressee labels exist. We additionally run with addressee edges removed to check that the deviation stream does not depend on gold metadata, which the file flags as unavailable in the wild.
Baselines
All graph models share the same frozen pretrained utterance encoder features, the same splits, the same hyperparameter search budget (equal number of trials per method) and the same number of training seeds. We report mean and spread over five seeds.
Naive reference. The do-nothing reference is the utterance encoder with a linear classifier and no graph. It tells us how much any structure adds.
Shallow homogeneous graph. DialogueGCN represents the "stay shallow, no speaker hub" position. It reports 58.10% F1 on MELD text and 64.18% average F1 on IEMOCAP. Beating it means improving overall weighted F1 and, specifically, within-speaker shift recall.
Heterogeneous hub model. ConGCN is the most direct hub-bearing competitor, reported at 59.4% multimodal weighted F1 on MELD. The file also quotes multimodal DialogueRNN at 60.25% weighted F in the MELD paper, while ConGCN's paper reports DialogueRNN at 57.0%. We will therefore rerun DialogueRNN under our protocol rather than trust either figure.
Strongest heterophily-aware method. HALO is the strongest existing method for heterophily-aware heterogeneous propagation. We port it to the same dialogue graph with one compatibility matrix per relation. To beat it, we must win on within-speaker shifts at equal parameter count.
Strongest spectral dialogue model. GS-MCC reports improvements of 3.3% W-Acc and 3.2% W-F1 over AdaGIN on IEMOCAP. For GS-MCC on MELD: no reported result found on Lacuna. We run its released configuration if available, and otherwise a faithful reimplementation, and report it as such.
Response generation. On Ubuntu IRC the baseline is HeterMPC with a BERT backbone, reported at BLEU-4 1.41 and ROUGE-L 11.20. Beating it means equal or better automatic scores together with higher within-speaker rank.
Ablations isolating the key ingredient. Three ablations isolate the separate deviation stream:
- Reparameterized skip: a residual plus speaker-mean model with the same parameters but no separate stream.
- Hub-routed deviation: the deviation stream allowed to flow through speaker nodes.
- Shared stream: a single stream with inverse-frequency speaker edges, as in ConGCN.
Why not just…?
Why not just down-weight or drop the speaker edges? This is the most serious threat. ConGCN already uses inverse speaking frequency, and dropping global speaker nodes in favor of dialogue-local ones may remove most of the collapse. The diagnosis compares global speaker nodes, dialogue-local speaker nodes, and additive embeddings, and the shared-stream ablation tests reweighting directly. Our position is that weakening speaker edges trades persona information against collapse, while separating the streams keeps both. If reweighting matches us on within-speaker shift recall, we will report that the problem is about node degree, and the contribution narrows to the diagnostic.
Why not just add residual connections and LayerNorm? The rank-measurement paper reports that these change feature scale but often do not prevent effective-rank collapse or rescue accuracy. That evidence comes from homogeneous benchmarks, so we test the point here with the reparameterized-skip ablation, which is the most direct version of this objection.
Why not just use a shallow window graph or a flattened transformer with speaker tags? Our claim is that collapse comes from hubs, not depth. Shallow models with speaker nodes therefore do not escape it, and models without speaker nodes avoid collapse only by giving up persona routing. HeterMPC reports outperforming fine-tuned BERT and BART on flattened Ubuntu IRC transcripts, which argues against flattening for multi-party settings. MELD and IEMOCAP dialogues are short, though, so a flattened encoder is a real competitor there. We include a fine-tuned flattened encoder with speaker prefixes under the same budget, and we accept the result either way.
Why not just use HALO's compatibility matrices? HALO already counters heterophily across relations. However, its speaker-to-utterance messages still blend into one state, which lets the hub average swamp turn-specific signal. If ported HALO matches us on shift subsets, the two-stream routing is unnecessary, and we will say so.
Proposed Experiments
Primary metrics. On MELD and IEMOCAP we report weighted F1 overall and on three subsets defined from gold labels:
- within-speaker shifts: the speaker's emotion differs from their previous turn;
- cross-speaker shifts: the emotion differs from the preceding turn by another speaker;
- non-shifts.
The MELD paper documents that models struggle at emotion shifts.
Decisive check 1: the effect lands where the mechanism says. Our method should improve within-speaker shifts significantly more than cross-speaker shifts. Its within-speaker effective rank should stay near the no-speaker-node variant, while the hub-routed ablation should drop.
Decisive check 2: depth. We sweep 1 to 8 layers. The hub hypothesis predicts the hub models' within-speaker rank drops already at 2 layers, independent of depth, while our model's curve is flat. If degradation in baselines appears only with depth, the hub framing is wrong, and the work reduces to a dialogue-specific depth study.
Decisive check 3: generation. On Ubuntu IRC we compare against HeterMPC on BLEU-4, ROUGE-L and within-speaker rank, with and without addressee edges. This tests whether the deviation stream helps where metadata is incomplete.
Failure outcome. A negative result on checks 1 and 2 would still be informative. It would show that the shallow depth of dialogue GNNs is not explained by over-smoothing.
References
- Modeling both Context- and Speaker-Sensitive Dependence for Emotion Detection in Multi-speaker Conversations
- HeterMPC: A Heterogeneous Graph Neural Network for Response Generation in Multi-Party Conversations
- DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation
- Complete the Missing Half: Augmenting Aggregation Filtering with Diversification for Graph Convolutional Networks
- Descent Steps of a Relation-Aware Energy Produce Heterogeneous Graph Neural Networks
- Revisiting Multimodal Emotion Recognition in Conversation from the Perspective of Graph Spectrum
- Are we measuring oversmoothing in graph neural networks correctly?
- MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations