Task-Routed Mixture-of-Experts with Cognitive Appraisal for Implicit Sentiment Analysis

Yaping Chai, Haoran Xie, S. Joe QinarXiv 2026 · DOI

TL;DR

This paper tackles the challenge of implicit sentiment analysis, where sentiment is inferred from context rather than explicit words. They propose a novel multi-task learning framework that leverages cognitive appraisal theory, adding auxiliary tasks for implicit sentiment detection and rationale generation. Their key innovation is a task-routed Mixture-of-Experts (MoE) architecture that allows tasks to share knowledge while maintaining distinct expert usage, significantly improving performance on implicit sentiment tasks.

What Problem Does This Paper Address?

Traditional Aspect-Based Sentiment Analysis (ABSA) often relies on explicit opinion words like "good" or "bad." However, many real-world scenarios involve implicit sentiment, where the sentiment is conveyed through descriptions of events or situations. For example, "We waited forty minutes before anyone came to the table" clearly expresses dissatisfaction with the service, but lacks explicit negative adjectives. This implicit sentiment analysis (ISA) is challenging because models need to infer sentiment from context. Existing models often focus solely on predicting the final sentiment label, providing limited guidance on why a particular sentiment is expressed. This paper argues that understanding the underlying cognitive appraisals (like goal inconduciveness or norm incompatibility) is crucial for robust ISA. They aim to build models that not only predict sentiment but also reason about its origin, bridging the gap between event descriptions and evaluative judgments.

Methods

The core of this work is a multi-task learning (MTL) framework built upon a sequence-to-sequence backbone (specifically, Flan-T5-large). They reformulate ISA into three text-to-text tasks:

  1. Polarity Classification (POL): The standard task of predicting positive, negative, or neutral sentiment.
  2. Implicit Sentiment Detection (IMP): Predicting whether the sentiment evidence is explicit or implicit.
  3. Cognitive Appraisal Reasoning (REA): Generating a free-form explanation of why the sentiment is expressed, drawing inspiration from cognitive appraisal theory. This is achieved using prompt-based generation with an LLM.

To address the common issue of task interference in MTL, they introduce a Task-Routed Mixture-of-Experts (MoE) architecture. Instead of a single feed-forward network (FFN) in transformer blocks, they replace a subset of FFNs with parallel "experts." A task-conditioned router, guided by the task identity (POL, IMP, or REA), selects a sparse mixture of these experts for each task. This allows tasks to share general linguistic knowledge while using specialized expert pathways.

Crucially, they propose a task-separated routing objective. This objective encourages the routing distributions for different tasks to be distinct, further reducing interference and promoting task specialization. The overall objective combines the standard generation loss for all three tasks with this routing separation loss.

Datasets

The experiments are conducted on two standard ABSA benchmarks:

  • SemEval-2014 Restaurant (Rest14)
  • SemEval-2014 Laptop (Lap14)

The authors also divide these datasets into explicit and implicit sentiment subsets based on whether sentiment is conveyed through explicit opinion words. No new benchmarks are introduced; instead, the focus is on improving performance on the implicit subset of existing datasets.

Key Insights and Ideas

  • Cognitive Appraisal Theory as a Guiding Principle: The paper successfully integrates cognitive appraisal theory into NLP, showing that understanding the underlying psychological evaluations (goals, expectations) can significantly improve sentiment analysis, especially for implicit cases.
  • Auxiliary Tasks are Crucial for Implicit Sentiment: Adding tasks like implicit sentiment detection and rationale generation provides richer supervision signals that help the model learn to infer sentiment from context, rather than just relying on explicit cues.
  • Task-Routed MoE for Effective MTL: The task-routed MoE architecture is a key innovation. It allows for efficient knowledge sharing across related tasks while enabling task-specific expert usage, mitigating the negative transfer often seen in standard MTL.
  • Routing Entropy Reveals Task Complexity: The analysis of routing entropy shows that simpler classification/detection tasks (POL, IMP) tend to use a smaller, more concentrated set of experts, while complex generation tasks (REA) benefit from broader expert mixing. This provides valuable insights into how MoE architectures can adapt to different task demands.
  • Balanced Routing Separation is Key: The hyperparameter $\lambda_{sep}$ (routing separation weight) is critical. Too little separation leads to oversharing and weakened task signals, while too much can overly constrain routing. An intermediate value (around 0.4) strikes a balance, leading to optimal performance.

Key Results

The proposed Task-Routed Mixture-of-Experts (TRMOE) model achieves state-of-the-art results on both the Rest14 and Lap14 datasets, particularly showing strong gains on the implicit sentiment subsets.

  • Overall Performance: The model achieves the best scores across various metrics (accuracy and F1-score) on both datasets compared to conventional ABSA baselines, inference-only LLMs, and other instruction-based fine-tuning methods. For instance, on Rest14, their model achieves 90.21% accuracy and 84.74% F1-score for "All A" (all aspects), and 75.90% for "ISA A" (implicit sentiment analysis).
  • Implicit Sentiment Improvement: The gains are most pronounced on the implicit sentiment analysis subset (ISA A), highlighting the effectiveness of their auxiliary tasks and MoE approach for inferring sentiment from context.
  • Ablation Studies: Removing multi-task learning (w/o MTL) significantly degrades performance, especially on implicit sentiment tasks, confirming the benefit of auxiliary supervision. Removing the MoE architecture (w/o MoE) also leads to performance drops, demonstrating the advantage of the sparse expert routing.

Limitations and Future Work

  • Hyperparameter Sensitivity: The performance is sensitive to the routing separation hyperparameter $\lambda_{sep}$, requiring careful tuning.
  • Expert Number: While they found an optimal number of experts (N=5) for their setup, the ideal number might vary across different tasks or architectures.
  • LLM for Rationale Generation: The use of a separate LLM (GPT-5.4-mini) for generating rationales is a component that could potentially be integrated more tightly or explored with different generation models.

Future work could explore more sophisticated routing mechanisms, investigate the interpretability of expert assignments across different tasks, and extend this framework to other NLP tasks that can benefit from cognitive appraisal and multi-task learning with sparse expert routing.

Summary updated

BibTeX
@misc{chai2026task,
  author = {Yaping Chai and Haoran Xie and S. Joe Qin},
  title = {{Task-Routed Mixture-of-Experts with Cognitive Appraisal for Implicit Sentiment Analysis}},
  year = {2026},
  eprint = {2605.20916},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.20916},
}
Figure