Echotune: A Modular Extractor Leveraging the Variable-Length Nature of Speech in ASR Tasks

Sizhou Chen, Songyang Gao, Sen FangarXiv 2023 · DOI · arXiv

Abstract

Automatic Speech Recognition (ASR) models often struggle with the hierarchical and variable-length nature of speech features, which range from millisecond-level frames to multi-second discourse. This paper introduces Echotune, a modular framework featuring Echo Multi-Scale Attention (Echo-MSA) and a Dual Focus Gate. By integrating these components, the model dynamically balances local and global context, achieving up to a 10.5% relative reduction in Word Error Rate (WER) on the LibriSpeech benchmark.

Introduction

While Transformer architectures have set new standards in ASR, their reliance on fixed-length attention windows presents a "one-size-fits-all" limitation. Speech is inherently hierarchical: phonemes, words, and sentences occupy vastly different temporal scales. Fixed windows often result in over-smoothing of fine-grained local details or the loss of long-range dependencies. Echotune addresses this by making the attention mechanism "elastic," allowing it to adapt to the specific temporal requirements of different speech units.

Architectural Innovations

The core of the Echotune framework consists of two primary modules designed to enhance the feature extraction process:

Echo Multi-Scale Attention (Echo-MSA)

Unlike standard self-attention, Echo-MSA utilizes a variable-length window ($W_\phi$) that can be tailored to different stages of the processing chain (e.g., frames, phonemes, or words). It leverages depthwise separable convolutions to process Query (Q), Key (K), and Value (V) vectors, significantly reducing computational overhead while maintaining high representational capacity.

Dual Focus Gate

To ensure that the model retains global context while gaining specialized local focus, the authors introduce the Dual Focus Gate. This parallel structure acts as a learned mixer, using ReLU and Sigmoid activations to dynamically weight the outputs of standard self-attention and the new Echo-MSA.

Architecture of the Echo-MSA and Dual Focus Gate modules.

Experimental Setup

The framework was evaluated using the LibriSpeech corpus, including the 100-hour "clean" subset for supervised training and the full 960-hour dataset for large-scale validation. Additionally, Libri-light (60,000 hours of unlabeled audio) supported self-supervised pre-training. Performance was measured across standard dev and test sets using Word Error Rate (WER).

Results and Discussion

Echotune demonstrated superior performance across various data scales and model sizes. Built upon a data2vec backbone, the "Base" version achieved a 2.4 WER on the LibriSpeech 100h clean set, a 7.7% relative improvement over the baseline. The "Large" 24-layer variant reached a 1.7 WER on test-clean, representing a 10.5% relative reduction.

Comparison of Echotune against state-of-the-art ASR models on LibriSpeech.

The model proved exceptionally robust in low-resource scenarios. With only 10 minutes of labeled data, Echotune outperformed the baseline by 4.1%, suggesting that its multi-scale inductive bias effectively compensates for data scarcity. Ablation studies further confirmed that the combination of Echo-MSA and the Dual Focus Gate is critical for these gains.

Conclusion

Echotune offers a flexible, modular approach to ASR by acknowledging and leveraging the multi-scale nature of speech. By complementing global attention with elastic, local multi-scale attention, the framework achieves state-of-the-art results while remaining efficient. Future work will focus on enhancing local acoustic feature utilization and testing the model's robustness against diverse accents and noisy environments.

Summary updated

BibTeX
@misc{chen2023echotune,
  author = {Sizhou Chen and Songyang Gao and Sen Fang},
  title = {{Echotune: A Modular Extractor Leveraging the Variable-Length Nature of Speech in ASR Tasks}},
  year = {2023},
  eprint = {2309.07765},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.07765},
}
Figure