Cross-Talk Reduction

Zhong-Qiu Wang, Anurag Kumar, Shinji Watanabe

Abstract

In multi-talker environments, close-talk microphones often capture "cross-talk" from nearby speakers, limiting their utility for data annotation and model evaluation. This paper introduces Cross-Talk Reduction (CTR) and proposes CTRnet, a neural separation model designed to cancel cross-talk in an unsupervised or weakly-supervised manner. By leveraging Forward Convolutive Prediction (FCP) and multi-microphone redundancy, CTRnet significantly outperforms traditional signal processing methods. On the CHiME-7 real-world dataset, CTRnet reduces Word Error Rate (WER) from 27.8% to 22.6%, demonstrating its effectiveness in challenging acoustic conditions.

Introduction: The Cross-Talk Problem

In settings like meetings or social gatherings, audio is typically captured via far-field microphone arrays and individual close-talk microphones (e.g., lapel mics). While close-talk mics are intended to capture only the wearer's voice, they frequently suffer from "cross-talk"—interference from other speakers in the room. This contamination prevents these recordings from serving as clean "ground truth" for training or evaluating far-field separation models.

The cross-talk problem in a reverberant room: close-talk microphones capture both the intended speaker's voice and interference from others.

The authors define the CTR task as the process of isolating the target speaker's voice from these contaminated close-talk recordings, effectively turning them into "dry" source signals.

Methodology: CTRnet and Forward Convolutive Prediction

CTRnet treats cross-talk reduction as a blind deconvolution problem. Unlike supervised models that require clean targets, CTRnet is trained using mixture constraints.

The CTRnet architecture uses a DNN to predict clean speech, which is then re-reverberated via FCP to match the original mixtures for unsupervised training.

Unsupervised Learning via Mixture Constraints

The core of CTRnet is a Deep Neural Network (DNN)—specifically the state-of-the-art TF-GridNet—which predicts an estimate for each speaker's "dry" speech. To train this without clean labels, the model employs Forward Convolutive Prediction (FCP). FCP learns linear filters that "re-reverberate" the predicted clean speech to match its appearance at every microphone in the environment. The training loss then minimizes the difference between the sum of these filtered estimates and the actual recorded mixtures.

Weak Supervision with Timestamps

To handle real-world complexities like silent periods, the authors incorporate speaker-activity timestamps (VAD). During training, the model's predictions are "muted" during a speaker's known silence. This simple regularization prevents "over-separation," where the model might incorrectly attribute noise or other speakers' voices to a silent target.

Experimental Evaluation

The researchers evaluated CTRnet on two distinct datasets:

  • SMS-WSJ-FF-CT: A simulated 2-speaker environment with a 6-mic far-field array and two close-talk mics.
  • CHiME-7 (DASR): A challenging real-world dataset of dinner party conversations featuring moving speakers and hardware synchronization issues.

Key Results and Insights

CTRnet demonstrated substantial improvements over baseline methods across both datasets.

  • Simulated Performance: On SMS-WSJ-FF-CT, CTRnet improved the Scale-Invariant Signal-to-Distortion Ratio (SI-SDR) from an unprocessed 14.7 dB to 26.5 dB. This performance significantly exceeds traditional spatial clustering and independent vector analysis (IVA) techniques.
  • Real-World Impact: On the CHiME-7 dataset, the primary metric was Automatic Speech Recognition (ASR) accuracy. CTRnet reduced the Word Error Rate (WER) from 27.8% (unprocessed) to 22.6%. Notably, it outperformed Guided Source Separation (GSS), a common industry standard, which achieved a WER of 26.6%.

The success of CTRnet stems from its ability to exploit multi-microphone redundancy. By processing far-field and close-talk signals together, the DNN gains multiple spatial perspectives of the same source, allowing the unsupervised loss to converge on physically accurate separations.

Conclusion and Future Work

CTRnet provides a robust framework for cleaning close-talk recordings, unlocking their potential for high-stakes audio tasks. While highly effective, future work will address challenges such as hardware "clock drift" between microphone arrays and the need for more adaptive filtering in highly dynamic environments with moving speakers.

Summary updated

BibTeX
@inproceedings{wang2024cross,
  author = {Zhong-Qiu Wang and Anurag Kumar and Shinji Watanabe},
  title = {{Cross-Talk Reduction}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=NRLDnVZ9tb},
}
Figure