MRRL: Modifying the Reference via Reinforcement Learning for Non-Autoregressive Joint Multiple Intent Detection and Slot Filling

Xuxin Cheng, Zhihong Zhu, Bowen Cao, Qichen Ye, Yuexian Zou

Abstract

With the rise of non-autoregressive approach, some non-autoregressive models for joint multiple intent detection and slot filling have obtained the promising inference speed. However, most existing SLU models (1) suffer from the multi-modality problem that leads to reference intents and slots may not be suitable for training; (2) lack of alignment between the correct predictions of the two tasks, which extremely limits the overall accuracy. Therefore, in this paper, we propose Modifying the Reference via Reinforcement Learning (MRRL), a novel method for multiple intent detection and slot filling, which introduces a modifier and employs reinforcement learning. Specifically, we try to provide the better training target for the non-autoregressive SLU model via modifying the reference based on the output of the non-autoregressive SLU model, and propose a suitability reward to ensure that the output of the modifier module could fit well with the output of the non-autoregressive SLU model and does not deviate too far from the reference. In addition, we also propose a compromise reward to realize a flexible trade-off between the two subtasks. Experiments on two multi-intent datasets and non-autoregressive baselines demonstrate that our MRRL could consistently improve the performance of baselines. More encouragingly, our best variant achieves new state-of-the-art results, outperforming the previous best approach by 3.6 overall accuracy on MixATIS dataset.

Introduction

Spoken Language Understanding (SLU) is a critical component of modern virtual assistants, typically involving two subtasks: Intent Detection and Slot Filling. While traditional autoregressive models achieve high accuracy by predicting tokens sequentially, they suffer from slow inference speeds. Non-autoregressive (NAR) models address this by predicting all outputs in parallel, but they face two significant challenges:

  1. The Multi-modality Problem: NAR models lack internal dependencies during decoding, making it difficult to handle tokens that could logically belong to multiple labels. Static "gold" references may not always be the optimal training targets for parallel decoders.
  2. Task Misalignment: Intent detection and slot filling are often optimized independently. A lack of coordination between these tasks can lead to cases where one is predicted correctly while the other fails, dragging down the critical "overall accuracy" metric.

The MRRL Framework

To address these issues, we propose Modifying the Reference via Reinforcement Learning (MRRL). This method introduces a "Modifier" module that dynamically adjusts the training targets (references) to be more "learnable" for the NAR model.

The architecture of the proposed MRRL framework, showing the interaction between the shared encoder, decoders, and the modifier module with its associated rewards.

Modifier Module and RL Rewards

The Modifier module consists of NAR decoders that take the original reference and the model's current predictions to produce a modified training target. This process is optimized using Reinforcement Learning (REINFORCE) with two key reward signals:

  • Suitability Reward ($R_{suit}$): This reward ensures the modified target is easy for the model to fit (low training loss) while maintaining semantic similarity to the original ground truth ($R_{fit}$ and $R_{sim}$).
  • Compromise Reward ($R_{com}$): This reward explicitly targets task misalignment by using the overall accuracy of the joint tasks as a signal, forcing the intent and slot decoders to coordinate their predictions.

Crucially, the Modifier is only utilized during the training phase. At inference time, the SLU model operates independently, preserving the speed advantages of non-autoregressive architectures.

Experimental Results

We evaluated MRRL on two standard multi-intent benchmarks: MixATIS (airline travel domain) and MixSNIPS (diverse personal assistant intents).

Main performance comparison on MixATIS and MixSNIPS datasets. MRRL consistently improves various NAR baselines.

As shown in the results, MRRL consistently improves the performance of several strong NAR baselines, including GL-GIN, Co-guiding Net, and ReLa-Net. Our best variant, ReLa-Net + MRRL, achieved a new state-of-the-art overall accuracy of 55.8% on MixATIS (a 3.6% improvement) and 79.3% on MixSNIPS.

Ablation studies confirmed the importance of the proposed rewards. Removing the Suitability Reward led to a 1.8% drop in overall accuracy on MixATIS, while removing the Compromise Reward resulted in a 2.8% decrease, highlighting the necessity of both dynamic target adjustment and cross-task coordination.

Conclusion

MRRL provides a novel paradigm for training NAR SLU models by acknowledging that static gold labels may not be the most effective training signals for parallel decoders. By dynamically modifying references through reinforcement learning, MRRL effectively bridges the gap between intent detection and slot filling without adding any inference overhead. Future work will explore applying this reference modification logic to other non-autoregressive tasks such as machine translation.

Summary updated

Abstract

With the rise of non-autoregressive approach, some non-autoregressive models for joint multiple intent detection and slot filling have obtained the promising inference speed. However, most existing SLU models (1) suffer from the multi-modality problem that leads to reference intents and slots may not be suitable for training; (2) lack of alignment between the correct predictions of the two tasks, which extremely limits the overall accuracy. Therefore, in this paper, we propose Modifying the Reference via Reinforcement Learning (MRRL), a novel method for multiple intent detection and slot filling, which introduces a modifier and employs reinforcement learning. Specifically, we try to provide the better training target for the non-autoregressive SLU model via modifying the reference based on the output of the non-autoregressive SLU model, and propose a suitability reward to ensure that the output of the modifier module could fit well with the output of the non-autoregressive SLU model and does not deviate too far from the reference. In addition, we also propose a compromise reward to realize a flexible trade-off between the two subtasks. Experiments on two multi-intent datasets and non-autoregressive baselines demonstrate that our MRRL could consistently improve the performance of baselines. More encouragingly, our best variant achieves new state-of-the-art results, outperforming the previous best approach by 3.6 overall accuracy on MixATIS dataset.
BibTeX
@inproceedings{cheng2023mrrl,
  author = {Xuxin Cheng and Zhihong Zhu and Bowen Cao and Qichen Ye and Yuexian Zou},
  title = {{MRRL: Modifying the Reference via Reinforcement Learning for Non-Autoregressive Joint Multiple Intent Detection and Slot Filling}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  url = {https://openreview.net/forum?id=cmXrxorp9l},
}
Figure