Models that recognize emotion in conversation usually treat a dialogue as a graph. Each utterance is a node, and edges link it to earlier turns by the same or other speakers. Each node's representation is then updated by averaging in messages from its neighbors. This works because emotions are contagious: a sad turn tends to follow sad turns. It fails at the turns that matter most. When a speaker suddenly flips from neutral to disgust or fear, every neighbor of that utterance is neutral, and averaging pulls the rare emotion back toward the crowd. We propose a different aggregation rule. Before taking in its neighbors, each utterance measures how much it disagrees with its conversational context, and it uses that disagreement to decide how much of the context to accept. Turns that fit their context absorb it. Turns that break from it keep their own evidence.
The opportunity is practical. The benchmarks in this area are dominated by the neutral class, with imbalance ratios above 30:1 in some corpora (CLTeam1 at SemEval-2024 Task 10). The field's usual remedies, which are reweighted losses, oversampling, and generated minority examples, all act on the classifier or on the data. None of them acts on the graph step where the minority signal is actually erased. If that step is the bottleneck, a small change to how neighbors are weighted should produce gains concentrated at emotion shifts. Those gains should be additive to loss-level fixes rather than a substitute for them.
Core Claim
Our claim is that minority-class errors in conversational emotion recognition come largely from turns where the emotion changes. At those turns, neighbor aggregation acts as a smoothing filter, and a loss applied at the output cannot undo information that was averaged away inside the graph layers. The fix we test is a gate that needs no labels. It is computed from the gap between an utterance's own features and a summary of its causal context, and it controls how much of the context enters the utterance's update. Neighbor weights inside that context are also changed: each neighbor gets an independent sigmoid weight instead of a softmax weight. Under softmax, the weights must sum to one, so a crowd of neutral neighbors necessarily soaks up most of the attention.
The mechanism is written as
$$h_i^{(l+1)} = g_i\,W_s h_i^{(l)} + (1-g_i)\sum_{j\in\mathcal{N}_i}\beta_{ij}\,W_{r(i,j)} h_j^{(l)},\qquad g_i=\sigma\big(u^\top |h_i^{(l)}-\bar c_i^{(l)}|\big),$$
where the symbols mean the following. $h_i^{(l)}$ is utterance $i$'s representation at layer $l$. $\mathcal{N}_i$ is its set of past neighbors. $\bar c_i^{(l)}$ is the plain mean of those neighbors. $\beta_{ij}=\sigma(a^\top[h_i\|h_j])$ is a per-neighbor weight that does not have to sum to one across neighbors. $W_{r(i,j)}$ is a projection that depends on the speaker relation (the same speaker or another speaker). $u$, $a$ and $W_s$ are learned.
The graph topology stays causal, meaning each utterance sees only earlier turns, so the model can still run on a live, streaming dialogue.
Relational learning theory motivates the design. Collective inference, which infers each node's label jointly with its neighbors' labels, pays off when labels are autocorrelated across edges (Relational Learning with One Network). An emotion shift is a local break in that autocorrelation, and the gate $g_i$ is a per-node estimate of whether the break is present. Rel-MOSS motivates the proposal: it proves that minority signals decay exponentially over message-passing layers when majority neighbors dominate. Its fix, Rel-Gate, weights messages by the predicted probability that they come from the minority class, and it is built for static database graphs. Our gate differs in two ways. It uses no class posteriors, so it avoids the cold-start problem that BAT faces when early predictions are noisy. And it runs on temporally directed dialogue graphs.
Research Program
The backbone comes from DAG-ERC, which represents a dialogue as a directed acyclic graph. Each turn receives edges from the same speaker's last turn and from all turns in between, and the model stays stable up to 12 layers where relational graph attention degrades. Its authors also report that it tends to default to neutral. DAG-ERC supplies the method: we keep its graph construction, its speaker-dependent projections and its encoder features. We replace only its aggregation step with the contrast-gated rule above. In that sense it also constrains the design, because its gated recurrent unit already gives nodes some ability to preserve themselves, and our gain must be measured over that.
DialogueGCN and FER-GCN motivate the smoothing concern. Both use softmax-style or learned-adjacency weighting. FER-GCN deliberately spreads features from strong "peak" frames to weak ones. That is the right behavior for well-formed expressions and the wrong behavior for a single rare turn surrounded by neutral ones.
The program has three parts. The first is a diagnostic study that tests whether the premise holds. We freeze trained baselines, then train linear probes to recover each utterance's own gold emotion from its representation at every layer. We report how recoverable the label is at each layer, split two ways: shift turns versus turns that continue the previous emotion, and minority versus majority classes. The premise holds if recoverability for minority shift turns falls with depth while recoverability for majority continuation turns does not. The second part is the model, which is dropped into DAG-ERC. The third part is a factorial study that crosses the aggregation rule with loss-level imbalance fixes. The prior-weighted focal loss of Chen et al. scales up the loss on hard examples and on rare classes. The classifier weight regularization of Le Moine et al. penalizes the output layer to keep the weights for rare classes from shrinking. Crossing these with the aggregation rule shows whether the two kinds of fix are additive or redundant.
Baselines
The naive reference point has two parts. One is always predicting neutral, which fixes the floor for weighted F1. The other is the utterance encoder alone, with no graph, trained with standard cross-entropy. This shows how much any context model adds at all.
The strongest existing structural method is DAG-ERC. The file reports its F1 as 68.03% on IEMOCAP and 63.65% on MELD (DAG-ERC). The strongest reported text-only result in the file is DVL-CER at 70.11% weighted F1 on IEMOCAP and 39.92% on EmoryNLP (Dual-View Learning). DVL-CER adds a separate network that models transitions between emotion distributions over a fixed window of four utterances. It is the most direct competitor on emotion shifts, because it attacks the same failure through a separate transition model rather than through aggregation. The strongest imbalance-aware conversational system is CBERL, which reports a weighted-average F1 of 66.8% on MELD and 69.2% on IEMOCAP (CBERL). Those numbers are multimodal and use generated minority features, so we report CBERL as a reference and do not claim a matched comparison against it. The file reports no macro-F1 or per-class recall for DAG-ERC or DVL-CER on MELD, so we will measure these metrics ourselves for every baseline.
All re-run systems use the same RoBERTa utterance features and the same data splits. Each gets the same hyperparameter search budget of 30 trials, and results are averaged over five seeds with the same number of layers. DAG-ERC with prior-weighted focal loss is the loss-level competitor. A transplanted Rel-Gate, in which messages are gated by predicted minority probability, is the competitor for posterior-based gating.
Three ablations isolate the key ingredient. The first replaces the contrast gate with an unconditional learned residual scalar. The second concatenates each utterance's raw features to the classifier input. The third keeps softmax neighbor weights but adds the gate. To count as a win, the full model must beat DAG-ERC plus focal loss on macro-F1 and on minority recall at shift turns. The margin must exceed the spread across seeds on MELD and on at least one other corpus, and weighted F1 on the neutral-heavy test sets must not get worse.
Why not just…?
Why not just reweight the loss? Loss reweighting is the most serious threat. Chen et al. won the MSP-Podcast challenge largely through prior-weighted focal loss, with a macro-F1 of 35.69%. Against that, QiMP found that weighted losses left recall on anger at 61.04%, and Le Moine et al. argue that reweighting alone does not prevent the weights for rare classes from collapsing. The factorial study settles the question. If focal loss closes the gap at shift turns, and the layer-wise probes show no loss of minority information inside the graph, then our premise is wrong and we will report that.
Why not just add a residual connection, or feed each utterance's own features to the classifier? A skip connection also preserves the node's own evidence and costs nothing, and SCSE-GFD uses skip connections for this purpose in fraud graphs. The difference is that a skip connection preserves self-information by the same amount everywhere, which also weakens the context that helps on continuation turns. MELD's analysis shows that context carries the decisive cue in most correct predictions (MELD). Ablations one and two test this directly. If either one matches the conditional gate, the conditional part of the idea is unnecessary.
Why not just generate synthetic minority examples? Generation looks promising at first. CBERL reports 10 to 20 point gains on fear and disgust using a generative adversarial network (GAN), a model trained to produce realistic synthetic features. But the evidence on data-level fixes points in different directions. Oversampling failed on disgust in CLTeam1. BAT shows that changing graph topology alone can beat rebalancing. And the GAN in CBERL generates isolated feature vectors without any conversational context. We treat generation as orthogonal to our method rather than as a replacement for it. We test it by adding feature-level mixing of minority examples to both DAG-ERC and our model.
Why not just prompt a large language model? In the cited shared tasks, zero-shot Mistral 7B scored 0.32 weighted F1 on EdiREF, against 0.43 for fine-tuned XLM-R (ISDS-NLP). That gap makes a prompted model a reference point here rather than a competitor.
Proposed Experiments
The main benchmarks are MELD, IEMOCAP, EmoryNLP and DailyDialog, all text-only. We add the Hinglish EdiREF set from SemEval-2024 Task 10 as a stress test of extreme imbalance. The best weighted F1 reported in the file on that set is 44%, from CLTeam1. We report weighted F1 for comparability with prior work, and we also report macro-F1, per-class recall, and F1 restricted to shift turns.
The experiment that decides the question is the probe study run on the factorial grid. The proposal is supported under three conditions. First, the baseline's minority shift-turn probe accuracy must fall with depth. Second, the contrast gate must flatten that curve. Third, the gains on shift turns must add to the gains from focal loss rather than overlap with them. A secondary check tests generality. We run the gate on FER-GCN-style frame graphs for AFEW 8.0, where FER-GCN reports 55.67% accuracy, to see whether brief expression changes in video behave like shifts in dialogue. That check is exploratory and does not bear on the main claim.
References
- CLTeam1 at SemEval-2024 Task 10: Large Language Model based ensemble for Emotion Detection in Hinglish
- Relational Learning with One Network: An Asymptotic Analysis
- Rel-MOSS: Towards Imbalanced Relational Deep Learning on Relational Databases
- Class-Imbalanced Graph Learning without Class Rebalancing
- Directed Acyclic Graph Network for Conversational Emotion Recognition
- DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation
- Video-based Facial Expression Recognition using Graph Convolutional Networks
- 1st Place Solution to Odyssey Emotion Recognition Challenge Task1: Tackling Class Imbalance Problem
- Speaker Attentive Speech Emotion Recognition
- Dual-View Learning for Conversational Emotion Recognition Through Context and Emotion-Shift Modeling
- Deep Imbalanced Learning for Multimodal Emotion Recognition in Conversations
- QiMP at SemEval-2025 Task 11: Optimizing Text-based Emotion Classification in English Beyond Traditional Methods
- Spectrum-Constrained and Skip-Enhanced Graph Fraud Detection: Addressing Heterophily in Fraud Detection with Spectral and Spatial Modeling
- MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations
- ISDS-NLP at SemEval-2024 Task 10: Transformer based neural networks for emotion recognition in conversations