TL;DR
This paper investigates how wearing face masks impacts automatic speaker recognition systems. The authors propose a method to compensate for the acoustic distortions introduced by masks by analyzing their transfer functions, showing that this compensation can improve recognition accuracy in mismatched conditions.
What Problem Does This Paper Address?
Automatic speaker recognition systems are typically trained on clean speech recorded under ideal conditions. However, real-world scenarios often involve degraded speech, such as recordings from crime scenes or medical environments where individuals might wear face masks. These masks alter the speech signal in complex ways, affecting both the speaker's production (e.g., changes in vocal effort) and the sound transmission through the mask material and shape. This paper aims to disentangle these effects and develop a method to mitigate the negative impact of face masks on speaker recognition performance, particularly in forensic applications where comparing masked speech to unmasked speech is common.
Methods
The core of the paper's methodology involves two main parts:
- Measuring Face Mask Acoustic Properties: The researchers used an artificial voice generator and a dummy head in an anechoic chamber to measure the acoustic transfer functions of four different types of face masks: a motorcycle helmet, a rubber mask, a surgical mask, and a scarf. This setup allowed them to capture the passive filtering effects of the masks on the speech signal.
- Compensating for Mask Effects in Speaker Recognition: For speaker recognition experiments, the authors employed the state-of-the-art i-vector method. They then introduced a compensation step for speech recorded under a face mask. This compensation involved a "direct inversion" of the measured magnitude transfer function of the specific mask used. Essentially, they applied an inverse filter to the speech signal's spectrum to counteract the mask's filtering effect before extracting acoustic features (MFCCs). This approach assumes the mask acts as a linear time-invariant filter and ignores the phase response.
Datasets
The study utilized a speech corpus previously collected for forensic automatic speaker recognition. This corpus includes speech from 8 male and 8 female speakers, recorded in both reading and spontaneous formats. Speech was recorded under four different face mask conditions (motorcycle helmet, rubber mask, surgical mask, scarf) and a no-mask condition. Recordings were made using three microphones simultaneously. For the speaker recognition experiments, the second session of the reading speech material was used for enrollment, and spontaneous speech segments were used for testing.
Key Insights and Ideas
- Masks Primarily Affect High Frequencies: The measurements of mask transfer functions revealed that most spectral distortion occurs in the higher frequency ranges, rather than lower frequencies.
- Mask Type Matters Significantly: Different masks introduce varying degrees and types of attenuation and even some frequency bands with increased radiation.
- Direct Inversion Offers Improvement: Applying a simple direct inversion of the mask's magnitude transfer function as a compensation step in the acoustic feature extraction pipeline can reduce the performance degradation caused by mismatched mask conditions.
- Passive Effects are Key: The study focuses on compensating for the passive acoustic filtering effects of the mask, acknowledging that active changes in speech production due to wearing a mask are a separate, more complex issue for future work.
Key Results
- Uncompensated Mismatch: When speaker models were trained and tested with different face masks (Table 1), significant drops in speaker identification rates were observed compared to matched conditions. For instance, when testing with a "No mask" condition against models trained with a "Surgeon mask," the accuracy dropped from 95.2% to 88.6%.
- Compensated Mismatch: With the proposed direct inversion compensation (Table 2), the performance degradation due to mismatched masks was generally reduced. For the same "No mask" vs. "Surgeon mask" scenario, compensation improved the accuracy from 88.6% to 91.2%.
- Marginal Improvements: While compensation helped, the paper notes that the improvements were marginal in some cases, especially when the speech signal is already band-limited, as is common in speaker recognition systems.
Limitations and Future Work
- Ignoring Phase Response: The compensation method only considers the magnitude of the mask's transfer function, ignoring its phase response. This is a simplification, and a more accurate treatment of the face mask transfer function is suggested for future work.
- Ignoring Active Effects: The study acknowledges that face masks can also influence speech production (active effects), which are not accounted for in this compensation method. Future research should address these active effects.
- Linear Time-Invariant Assumption: The assumption that a face mask acts as a linear time-invariant filter might not hold true for all mask types or all speech conditions.
- Further Research: The authors suggest that more accurate compensation techniques for the face mask transfer function are needed, potentially incorporating phase information and addressing the active effects of mask wearing.
Summary updated