M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge

Fan Yu, Shiliang Zhang, Yihui Fu, Lei Xie, Siqi Zheng, Zhihao Du, Weilong Huang, Pengcheng Guo, Zhijie Yan, Bin Ma, Xin Xu, Hui Bu

TL;DR

The M2MeT challenge introduces AliMeeting, a new 120-hour Mandarin meeting corpus designed to tackle the messy reality of multi-party speech transcription. By providing synchronized far-field and near-field audio with high speaker overlap ratios, the dataset establishes a rugged new benchmark for speaker diarization and multi-speaker automatic speech recognition (ASR).

What Problem Does This Paper Address?

Automating meeting transcription—answering the question of "who spoke what, and when"—is one of the hardest frontiers in speech processing. While single-speaker ASR has achieved impressive accuracy in quiet environments, real-world meetings are chaotic. They feature an unknown number of participants, unconstrained speaking styles, background noise, reverberation, and, most critically, overlapping speech (people talking over each other).

A major bottleneck preventing progress in this field has been the lack of large, publicly available, heavily annotated datasets. Furthermore, almost all existing meeting corpora are in English. To bridge this gap, the authors introduce the AliMeeting corpus and launch the Multi-channel Multi-party Meeting Transcription Challenge (M2MeT) at ICASSP 2022.

Methods

The paper outlines baseline systems for two distinct challenge tracks:

  1. Speaker Diarization (Track 1): The authors deploy a Kaldi-based baseline derived from the CHiME-6 challenge. It uses a ResNet-based speaker embedding extractor trained on the CN-Celeb dataset, followed by Probabilistic Linear Discriminant Analysis (PLDA) scoring. The final speaker clusters are resolved using Agglomerative Hierarchical Clustering (AHC) and refined via a Variational Bayesian HMM (VBx) model.
  2. Multi-Speaker ASR (Track 2): For speech recognition, the authors utilize an end-to-end Transformer architecture enhanced with Conformer encoder blocks to capture both local and global context. To handle overlapping speech without hitting a scaling wall, they employ Serialized Output Training (SOT). Instead of predicting text for multiple speakers simultaneously via complex permutation matching, SOT serializes the transcripts of different speakers chronologically, separating them with a special "speaker change" token (<sc>).

For multi-channel far-field audio, a Context-clipping Conformal Differential Microphone Array (CDDMA) beamformer is used as a front-end enhancement step.

Datasets

The primary contribution is the AliMeeting corpus, which contains 118.75 hours of speech data split into Train (104.75h), Eval (4h), and Test (10h) sets.

  • The Setup: Data was collected across 13 different meeting rooms ranging from 8 to 55 $m^2$, featuring diverse acoustic properties and natural indoor noises (keyboard clicks, chair movements, etc.).
  • The Hardware: Audio was captured simultaneously via an 8-channel directional microphone array (far-field) and individual participant headsets (near-field).
  • The Participants: 481 native Mandarin speakers with balanced gender representation.

A key differentiator of AliMeeting is its high density of conversational overlap; in 4-speaker meetings, the speaker overlap ratio reaches nearly 60%.

Fig. 1. An example of recording venue and the topology of micro- phone array.

Key Insights and Ideas

  • The Diarization Overlap Blindspot: Standard diarization baselines assume a "one speaker per frame" rule. Because AliMeeting features heavy conversational overlap, the baseline's error rate skyrockets when evaluated strictly (at a 0-second collar allowance). This highlights a critical need for algorithms that explicitly detect and separate overlapping speech.
  • The Beamformer Paradox: While front-end beamforming (spatial filtering) is highly beneficial for ASR because it cleans up the target audio, the authors found it can actually degrade standard diarization performance. By suppressing secondary speakers, the beamformer inadvertently masks the very overlaps the diarization system needs to detect.
  • Cross-Domain Augmentation Wins: For ASR, training strictly on far-field data yields poor results due to room acoustics. However, mixing in clean, near-field headset data (Train-Ali-near) drastically improves the model's acoustic robustness, acting as an effective regularizer.

Key Results

Speaker Diarization

The baseline system achieved a Diarization Error Rate (DER) of 15.24% on the far-field evaluation set when using a standard 0.25-second collar. However, when evaluated with a strict 0-second collar, the DER jumped to 24.52%, driven heavily by missed speech during speaker overlaps.

Table 3. Speaker diarization results of meeting session with different number of speaker on Eval set (DER%).

Multi-Speaker ASR

Using the Serialized Output Training (SOT) Conformer model yielded massive improvements over single-speaker baselines. When the SOT model was trained on a combination of far-field and near-field data and paired with a front-end beamformer (SOTB_bf), it achieved the lowest Character Error Rate (CER) of 29.7% on the beamformed evaluation set.

Table 5. Multi-speaker ASR results on Eval set (CER%).

Limitations and Future Work

The primary limitation lies within the baseline diarization system itself: it is fundamentally incapable of assigning multiple speaker labels to a single frame of audio. Consequently, its performance plummets during heavy multi-talker discussions.

The authors suggest that future challenge participants focus on developing end-to-end neural diarization (EEND) systems capable of overlapping speaker detection, as well as tighter joint integration between front-end speech separation and back-end ASR.

Summary updated

Abstract

Recent development of speech processing, such as speech recognition, speaker diarization, etc., has inspired numerous applications of speech technologies. The meeting scenario is one of the most valuable and, at the same time, most challenging scenarios for the deployment of speech technologies. Specifically, two typical tasks, speaker diarization and multi-speaker automatic speech recognition have attracted much attention recently. However, the lack of large public meeting data has been a major obstacle for the advancement of the field. Therefore, we make available the AliMeeting corpus, which consists of 120 hours of recorded Mandarin meeting data, including far-field data collected by 8-channel microphone array as well as near-field data collected by headset microphone. Each meeting session is composed of 2-4 speakers with different speaker overlap ratio, recorded in rooms with different size. Along with the dataset, we launch the ICASSP 2022 Multi-channel Multi-party Meeting Transcription Challenge (M2MeT) with two tracks, namely speaker diarization and multi-speaker ASR, aiming to provide a common testbed for meeting rich transcription and promote reproducible research in this field. In this paper we provide a detailed introduction of the AliMeeting dateset, challenge rules, evaluation methods and baseline systems.
BibTeX
@inproceedings{yu2022m,
  author = {Fan Yu and Shiliang Zhang and Yihui Fu and Lei Xie and Siqi Zheng and Zhihao Du and Weilong Huang and Pengcheng Guo and Zhijie Yan and Bin Ma and Xin Xu and Hui Bu},
  title = {{M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2022},
  doi = {10.1109/icassp43922.2022.9746465},
}
Figure