A CRNN-GRU Based Reinforcement Learning Approach to Audio Captioning

Xuenan Xu, Heinrich Dinkel, Mengyue Wu, Kai YuDCASE 2020 · OpenReview

Abstract

Audio captioning involves generating descriptive natural language for acoustic environments. This work introduces a lightweight (5M parameter) framework consisting of a 5-layer CRNN encoder and a GRU decoder. By implementing a two-stage training process—standard cross-entropy followed by reinforcement learning via Self-Critical Sequence Training (SCST)—the authors mitigate exposure bias and align the model's objective with human-centric evaluation metrics. The approach demonstrated top-tier performance in the DCASE 2020 challenge, ranking 4th overall while maintaining high computational efficiency.

Introduction

Unlike sound event detection, which identifies discrete labels, audio captioning requires a holistic understanding of temporal and spectral relationships to produce fluent descriptions. Current research faces two primary obstacles: exposure bias, where models fail to recover from their own errors during inference because they were trained only on ground-truth tokens, and objective mismatch, where minimizing cross-entropy loss does not necessarily maximize evaluation metrics like CIDEr or BLEU.

System Architecture

The proposed model utilizes an encoder-decoder structure designed for both efficiency and representational power, totaling only 5 million parameters.

Encoder: 5-Layer CRNN

The encoder processes log-Mel spectrograms through five convolutional blocks to extract local frequency features. These are followed by a Bidirectional GRU (BiGRU) layer to capture temporal dependencies across the audio clip. A Global Average Pooling (GAP) layer then compresses the output into a fixed 256-dimensional audio embedding.

Decoder: GRU

The decoder is a single-layer GRU with 512 units. It generates captions word-by-word by attending to the audio embedding and the previously generated token, as shown in the architecture overview below.

The proposed encoder-decoder architecture showing the 5-layer CRNN encoder and the GRU decoder with reinforcement learning optimization.

Training Methodology

The training is conducted in two distinct stages to bridge the gap between language modeling and evaluation metrics:

  1. Cross-Entropy (XE) Training: The model first learns basic language structure and audio-text alignment using standard teacher forcing.
  2. Reinforcement Learning (RL) Fine-tuning: Using Self-Critical Sequence Training (SCST), the model generates sequences and receives a "reward" based on the CIDEr score. This stage forces the model to rely on its own predictions during training, effectively reducing exposure bias and optimizing directly for the metrics used in competition.

Experimental Results

Evaluated on the Clotho dataset (4,981 audio samples), the CRNN-based models significantly outperformed GRU-only baselines. The inclusion of reinforcement learning and data augmentation proved critical for high performance.

Performance comparison of various model configurations against baseline and challenge-winning systems.

Key findings from the experimental results include:

  • Impact of RL: Fine-tuning with SCST (CRNN-R) improved the SPIDEr score from 0.142 to 0.194 compared to the base CRNN model.
  • Data Augmentation: Incorporating SpecAugment (CRNN-R-SpecAug) further boosted the SPIDEr score to 0.223, outperforming many complex ensemble models.
  • Efficiency: The system achieves competitive results with only 5 million parameters, making it highly suitable for resource-constrained environments compared to other DCASE entries using over 30 million parameters.

Conclusion

The combination of a CRNN encoder and RL-based fine-tuning provides a robust and efficient solution for audio captioning. While the model occasionally favors general vocabulary (e.g., "running") over highly specific human descriptors (e.g., "vibrating"), its ability to optimize directly for evaluation metrics makes it a powerful and lightweight architecture for automated audio description.

Summary updated

BibTeX
@inproceedings{xu2020crnn,
  author = {Xuenan Xu and Heinrich Dinkel and Mengyue Wu and Kai Yu},
  title = {{A CRNN-GRU Based Reinforcement Learning Approach to Audio Captioning}},
  booktitle = {DCASE},
  year = {2020},
  url = {https://openreview.net/forum?id=aafgXBaqOl},
}
Figure