Temporally Correlated Task Scheduling for Sequence Learning

Xueqing Wu, Lewen Wang, Yingce Xia, Weiqing Liu, Lijun Wu, Shufang Xie, Tao Qin, Tie-Yan Liu

Introduction

Sequence learning models often tackle a primary task that is surrounded by a family of "temporally correlated" auxiliary tasks. For instance, in simultaneous machine translation—where the goal is to translate words as they arrive—a "wait-3" task (waiting for 3 source words before translating) is temporally related to "wait-1" or "wait-7" tasks. Similarly, in stock forecasting, predicting tomorrow's price is inherently linked to predicting the price two days from now.

While researchers recognize that these auxiliary tasks can help the main model generalize better, the challenge lies in knowing which tasks to use and when to apply them during training. Standard multi-task learning (MTL) treats all tasks equally, and curriculum learning (CL) relies on fixed, pre-defined schedules. Neither approach adapts to the specific needs of a training sample or the evolving state of the model. To address this, researchers have introduced a learnable task scheduler that adaptively selects auxiliary training tasks to boost the performance of a primary sequence learning model.

A Learnable Task Scheduler

The core of the proposed method is a learnable task scheduler ($\phi$) that acts as a meta-controller. The training process is framed as a bi-level optimization problem and solved using reinforcement learning (RL).

  1. Bi-level Optimization: The training is split into two interacting loops. The "inner loop" updates the main model's parameters ($\theta$) using auxiliary tasks selected by the scheduler. The "outer loop" updates the scheduler's parameters ($\omega$) based on how well the main model performs on a validation set.
  2. Reinforcement Learning: The scheduler functions as a policy network. For every training minibatch, it "acts" by picking an auxiliary task. At the end of an episode, it receives a "reward" based on the validation performance improvement on the main task, trained via the REINFORCE algorithm.
  3. State Representation: To make informed decisions, the scheduler evaluates a 7-dimensional feature vector that includes the current training loss, historical losses, validation loss, and the training progress (step ratio).
The optimization workflow of one episode, showing the interaction between the scheduler and the model.

Key Insights and Dynamic Assistance

By dynamically scheduling tasks, the model exhibits several intelligent behaviors:

  • Dynamic Assistance: The scheduler learns that the optimal auxiliary task changes over time. Early in training, it tends to pick "easier" tasks (e.g., translation with higher latency) to provide a stable learning signal, later shifting to more complex tasks as the model matures.
  • Sample-Specific Scheduling: Because the scheduler observes features of the current data, it can assign different auxiliary tasks to different sentences, effectively acting as intelligent data augmentation.
  • The "Aha" Moment: Strategy analysis revealed an interesting pattern: for a "wait-3" translation task, the scheduler eventually preferred "wait-12" as an auxiliary task. This suggests that training on high-latency (easier) tasks provides a much stronger representation for low-latency (harder) tasks than training on the hard task alone.

Experimental Results

The researchers evaluated their Temporally Correlated Task Scheduling (TCTS) method on two distinct domains: Simultaneous Machine Translation (using IWSLT and WMT datasets) and Stock Trend Forecasting (using CSI300 index data).

TCTS consistently outperformed vanilla baselines, MTL, and CL approaches.

  • Translation: On the IWSLT En→Vi task, TCTS improved the BLEU score (a standard measure of translation quality) by 1 to 3 points over the standard wait-$k$ baseline. Across various latency constraints, the TCTS method consistently achieved superior translation quality.
Translation quality (BLEU) vs. latency (AP/AL) on IWSLT tasks. The red line represents the proposed TCTS method.
  • Stock Forecasting: TCTS achieved a higher RankIC (correlation between predicted and actual moves) and lower Mean Square Error (MSE) compared to strong baselines like GRU, MLP, and LightGBM.
  • Efficiency: Despite the inherent complexity of bi-level optimization, the additional training overhead was only 20%–30%. The authors argue this is a highly favorable trade-off given the significant accuracy gains.

Limitations and Future Directions

While highly effective, the method has a few limitations. First, while it improves accuracy under a given latency constraint, it does not explicitly attempt to minimize the latency itself during training. Second, in domains like stock forecasting, introducing too many auxiliary tasks (e.g., a set of 10 tasks) can sometimes introduce noise that degrades performance, indicating a limit to how much temporal correlation can be exploited.

Looking forward, the authors plan to extend this scheduling logic to other sequence tasks, such as action prediction in video, streaming speech recognition, and weather forecasting. They also aim to integrate the scheduler with more advanced architectures like Graph Attention Networks (GAT) to further push the boundaries of sequence learning.

Summary updated

Abstract

Sequence learning has attracted much research attention from the machine learning community in recent years. In many applications, a sequence learning task is usually associated with multiple tempo...
BibTeX
@inproceedings{wu2021temporally,
  author = {Xueqing Wu and Lewen Wang and Yingce Xia and Weiqing Liu and Lijun Wu and Shufang Xie and Tao Qin and Tie-Yan Liu},
  title = {{Temporally Correlated Task Scheduling for Sequence Learning}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
  url = {https://openreview.net/forum?id=FynX8He2_uL},
}
Figure