TL;DR
The M2MeT challenge introduces AliMeeting, a new 120-hour Mandarin meeting corpus designed to tackle the messy reality of multi-party speech transcription. By providing synchronized far-field and near-field audio with high speaker overlap ratios, the dataset establishes a rugged new benchmark for speaker diarization and multi-speaker automatic speech recognition (ASR).
What Problem Does This Paper Address?
Automating meeting transcription—answering the question of "who spoke what, and when"—is one of the hardest frontiers in speech processing. While single-speaker ASR has achieved impressive accuracy in quiet environments, real-world meetings are chaotic. They feature an unknown number of participants, unconstrained speaking styles, background noise, reverberation, and, most critically, overlapping speech (people talking over each other).
A major bottleneck preventing progress in this field has been the lack of large, publicly available, heavily annotated datasets. Furthermore, almost all existing meeting corpora are in English. To bridge this gap, the authors introduce the AliMeeting corpus and launch the Multi-channel Multi-party Meeting Transcription Challenge (M2MeT) at ICASSP 2022.
Methods
The paper outlines baseline systems for two distinct challenge tracks:
- Speaker Diarization (Track 1): The authors deploy a Kaldi-based baseline derived from the CHiME-6 challenge. It uses a ResNet-based speaker embedding extractor trained on the CN-Celeb dataset, followed by Probabilistic Linear Discriminant Analysis (PLDA) scoring. The final speaker clusters are resolved using Agglomerative Hierarchical Clustering (AHC) and refined via a Variational Bayesian HMM (VBx) model.
- Multi-Speaker ASR (Track 2): For speech recognition, the authors utilize an end-to-end Transformer architecture enhanced with Conformer encoder blocks to capture both local and global context. To handle overlapping speech without hitting a scaling wall, they employ Serialized Output Training (SOT). Instead of predicting text for multiple speakers simultaneously via complex permutation matching, SOT serializes the transcripts of different speakers chronologically, separating them with a special "speaker change" token (
<sc>).
For multi-channel far-field audio, a Context-clipping Conformal Differential Microphone Array (CDDMA) beamformer is used as a front-end enhancement step.
Datasets
The primary contribution is the AliMeeting corpus, which contains 118.75 hours of speech data split into Train (104.75h), Eval (4h), and Test (10h) sets.
- The Setup: Data was collected across 13 different meeting rooms ranging from 8 to 55 $m^2$, featuring diverse acoustic properties and natural indoor noises (keyboard clicks, chair movements, etc.).
- The Hardware: Audio was captured simultaneously via an 8-channel directional microphone array (far-field) and individual participant headsets (near-field).
- The Participants: 481 native Mandarin speakers with balanced gender representation.
A key differentiator of AliMeeting is its high density of conversational overlap; in 4-speaker meetings, the speaker overlap ratio reaches nearly 60%.
Key Insights and Ideas
- The Diarization Overlap Blindspot: Standard diarization baselines assume a "one speaker per frame" rule. Because AliMeeting features heavy conversational overlap, the baseline's error rate skyrockets when evaluated strictly (at a 0-second collar allowance). This highlights a critical need for algorithms that explicitly detect and separate overlapping speech.
- The Beamformer Paradox: While front-end beamforming (spatial filtering) is highly beneficial for ASR because it cleans up the target audio, the authors found it can actually degrade standard diarization performance. By suppressing secondary speakers, the beamformer inadvertently masks the very overlaps the diarization system needs to detect.
- Cross-Domain Augmentation Wins: For ASR, training strictly on far-field data yields poor results due to room acoustics. However, mixing in clean, near-field headset data (Train-Ali-near) drastically improves the model's acoustic robustness, acting as an effective regularizer.
Key Results
Speaker Diarization
The baseline system achieved a Diarization Error Rate (DER) of 15.24% on the far-field evaluation set when using a standard 0.25-second collar. However, when evaluated with a strict 0-second collar, the DER jumped to 24.52%, driven heavily by missed speech during speaker overlaps.
Multi-Speaker ASR
Using the Serialized Output Training (SOT) Conformer model yielded massive improvements over single-speaker baselines. When the SOT model was trained on a combination of far-field and near-field data and paired with a front-end beamformer (SOTB_bf), it achieved the lowest Character Error Rate (CER) of 29.7% on the beamformed evaluation set.
Limitations and Future Work
The primary limitation lies within the baseline diarization system itself: it is fundamentally incapable of assigning multiple speaker labels to a single frame of audio. Consequently, its performance plummets during heavy multi-talker discussions.
The authors suggest that future challenge participants focus on developing end-to-end neural diarization (EEND) systems capable of overlapping speaker detection, as well as tighter joint integration between front-end speech separation and back-end ASR.
Summary updated