Abstract
In multi-talker environments, close-talk microphones often capture "cross-talk" from nearby speakers, limiting their utility for data annotation and model evaluation. This paper introduces Cross-Talk Reduction (CTR) and proposes CTRnet, a neural separation model designed to cancel cross-talk in an unsupervised or weakly-supervised manner. By leveraging Forward Convolutive Prediction (FCP) and multi-microphone redundancy, CTRnet significantly outperforms traditional signal processing methods. On the CHiME-7 real-world dataset, CTRnet reduces Word Error Rate (WER) from 27.8% to 22.6%, demonstrating its effectiveness in challenging acoustic conditions.
Introduction: The Cross-Talk Problem
In settings like meetings or social gatherings, audio is typically captured via far-field microphone arrays and individual close-talk microphones (e.g., lapel mics). While close-talk mics are intended to capture only the wearer's voice, they frequently suffer from "cross-talk"—interference from other speakers in the room. This contamination prevents these recordings from serving as clean "ground truth" for training or evaluating far-field separation models.
The authors define the CTR task as the process of isolating the target speaker's voice from these contaminated close-talk recordings, effectively turning them into "dry" source signals.
Methodology: CTRnet and Forward Convolutive Prediction
CTRnet treats cross-talk reduction as a blind deconvolution problem. Unlike supervised models that require clean targets, CTRnet is trained using mixture constraints.
Unsupervised Learning via Mixture Constraints
The core of CTRnet is a Deep Neural Network (DNN)—specifically the state-of-the-art TF-GridNet—which predicts an estimate for each speaker's "dry" speech. To train this without clean labels, the model employs Forward Convolutive Prediction (FCP). FCP learns linear filters that "re-reverberate" the predicted clean speech to match its appearance at every microphone in the environment. The training loss then minimizes the difference between the sum of these filtered estimates and the actual recorded mixtures.
Weak Supervision with Timestamps
To handle real-world complexities like silent periods, the authors incorporate speaker-activity timestamps (VAD). During training, the model's predictions are "muted" during a speaker's known silence. This simple regularization prevents "over-separation," where the model might incorrectly attribute noise or other speakers' voices to a silent target.
Experimental Evaluation
The researchers evaluated CTRnet on two distinct datasets:
- SMS-WSJ-FF-CT: A simulated 2-speaker environment with a 6-mic far-field array and two close-talk mics.
- CHiME-7 (DASR): A challenging real-world dataset of dinner party conversations featuring moving speakers and hardware synchronization issues.
Key Results and Insights
CTRnet demonstrated substantial improvements over baseline methods across both datasets.
- Simulated Performance: On SMS-WSJ-FF-CT, CTRnet improved the Scale-Invariant Signal-to-Distortion Ratio (SI-SDR) from an unprocessed 14.7 dB to 26.5 dB. This performance significantly exceeds traditional spatial clustering and independent vector analysis (IVA) techniques.
- Real-World Impact: On the CHiME-7 dataset, the primary metric was Automatic Speech Recognition (ASR) accuracy. CTRnet reduced the Word Error Rate (WER) from 27.8% (unprocessed) to 22.6%. Notably, it outperformed Guided Source Separation (GSS), a common industry standard, which achieved a WER of 26.6%.
The success of CTRnet stems from its ability to exploit multi-microphone redundancy. By processing far-field and close-talk signals together, the DNN gains multiple spatial perspectives of the same source, allowing the unsupervised loss to converge on physically accurate separations.
Conclusion and Future Work
CTRnet provides a robust framework for cleaning close-talk recordings, unlocking their potential for high-stakes audio tasks. While highly effective, future work will address challenges such as hardware "clock drift" between microphone arrays and the need for more adaptive filtering in highly dynamic environments with moving speakers.
Summary updated