The ISCSLP 2022 Intelligent Cockpit Speech Recognition Challenge (ICSRC): Dataset, Tracks, Baseline and Results

Ao Zhang, Fan Yu, Kaixun Huang, Lei Xie, Longbiao Wang, Eng Siong Chng, Hui Bu, Binbin Zhang, Wei Chen, Xin Xu

Abstract

Automatic Speech Recognition (ASR) within the automotive environment faces unique challenges, including complex acoustic noise and the high frequency of named entities in voice commands. The ISCSLP 2022 Intelligent Cockpit Speech Recognition Challenge (ICSRC) was established to benchmark state-of-the-art ASR systems in these demanding conditions. By providing a specialized 20-hour in-vehicle Mandarin dataset, the challenge facilitated the development of robust models that significantly outperformed standard baselines. This paper summarizes the challenge's dataset, the strategies employed by top-performing teams—such as Conformer architectures and N-gram language model fusion—and the final results across two competitive tracks.

Introduction

As voice interaction becomes a primary interface in modern "intelligent cockpits," the demand for high-accuracy ASR systems has surged. However, the vehicle cabin is a hostile environment for speech processing. Unlike quiet home environments, car cabins feature irregular geometries that cause unique reverberations, compounded by constant noise from engines, tires, wind, and infotainment systems. Furthermore, cockpit commands are linguistically dense with named entities, such as specific Points of Interest (POI), contact names, and song titles, which are often underrepresented in general-purpose training data.

The ICSRC 2022 was designed to address these gaps by providing a realistic dataset and a platform for testing specialized ASR techniques. The challenge focused on two primary tracks: Track I for limited-parameter models (<15M) suitable for on-device deployment, and Track II for unlimited-parameter models typical of cloud-based systems.

The ICSRC Dataset

A central contribution of this challenge is the introduction of a new 20-hour Mandarin Chinese dataset recorded inside a New-Energy Vehicle (NEV). The data was captured via a Hi-Fi microphone mounted on the dashboard, with speakers positioned in the passenger seat to simulate realistic interaction scenarios.

The dataset covers a wide range of environmental conditions, including:

  • Road Types: City streets and highways.
  • Vehicle States: Varying speeds and window configurations (open vs. closed).
  • Content Categories: Commands for air conditioning, phone calls, music playback, navigation (POI), and general conversational "others."

While the 20-hour set is relatively small for training end-to-end (E2E) systems from scratch, it served as a critical fine-tuning resource (10 hours for evaluation and 11 hours for final testing) for models pre-trained on larger open-source corpora like AISHELL-1.

Methodological Approaches

Top-performing teams moved beyond standard E2E models, employing a suite of advanced techniques to handle the cabin's acoustic and linguistic complexities.

1. Advanced Architectures

Most participants adopted Conformer architectures, which leverage the strengths of both Transformers (for global context) and Convolutional Neural Networks (for local feature extraction). The winning team (T044) utilized a CTC-only model for the parameter-constrained Track I to maximize efficiency, while deploying a massive 574M parameter model for Track II.

2. N-Gram Language Model Fusion

Despite the trend toward pure neural E2E systems, the challenge demonstrated that traditional 6-gram language models remain essential for domain-specific tasks. Winners used iterative strategies to build language models that specifically simulated in-domain texts, significantly improving the recognition of rare POIs and contact names that neural models often struggle to predict.

3. Knowledge Distillation

To bridge the performance gap between large and small models, teams utilized "teacher-student" training. In this setup, the high-performing, unlimited-parameter models from Track II provided "soft labels" to guide the training of the smaller Track I models, allowing the compact systems to inherit the nuances of their larger counterparts.

4. Speech Enhancement and Augmentation

Several teams integrated front-end denoising models, such as DCCRN, to clean audio signals before ASR processing. Additionally, Text-to-Speech (TTS) was used as a data augmentation tool to generate synthetic training examples, helping to mitigate the scarcity of labeled in-vehicle data.

Results and Analysis

The challenge saw a dramatic reduction in Character Error Rates (CER) compared to the official baseline, which was a standard Conformer trained on AISHELL-1.

  • Track I (Limited): The winning entry achieved a 10.66% CER, a massive improvement over the baseline's 53.57%.
  • Track II (Unlimited): The winner reached 8.94% CER, compared to the baseline's 47.98%.

An analysis of the results revealed that while fixed-phrase commands (e.g., "Air Conditioner") were easily mastered (CER ~1.5%), the "POI" and "Phone Call" categories remained difficult (CER ~12-21%) due to the infinite variety of names and locations. Interestingly, the 40x increase in model size between Track I and Track II only yielded a ~1.7% absolute improvement in CER, suggesting that for this specific task, data augmentation and language model fusion are more impactful than raw parameter count.

Conclusion and Future Work

The ISCSLP 2022 ICSRC highlighted the effectiveness of hybrid approaches—combining modern neural architectures with traditional linguistic modeling—to solve the unique hurdles of in-car ASR. However, the challenge also identified ongoing limitations, particularly the difficulty of recognizing rare named entities and the constraints of single-channel audio.

Future iterations of the challenge are expected to explore multi-channel audio for better speaker isolation, multi-modal integration (such as lip-reading) to combat extreme noise, and speaker diarization to better manage full-duplex conversations between drivers and passengers.

Summary updated

BibTeX
@inproceedings{zhang2022iscslp,
  author = {Ao Zhang and Fan Yu and Kaixun Huang and Lei Xie and Longbiao Wang and Eng Siong Chng and Hui Bu and Binbin Zhang and Wei Chen and Xin Xu},
  title = {{The ISCSLP 2022 Intelligent Cockpit Speech Recognition Challenge (ICSRC): Dataset, Tracks, Baseline and Results}},
  booktitle = {ISCSLP},
  year = {2022},
  doi = {10.1109/iscslp57327.2022.10037868},
}
Figure