Introduction
Speech Activity Detection (SAD) is the critical first step in many audio processing pipelines, responsible for distinguishing human speech from background noise. While this task is relatively straightforward in controlled environments, it becomes exceptionally difficult in "naturalistic" settings. This paper addresses the extreme challenges posed by the Fearless Steps Challenge, which utilizes digitized analog communications from NASA’s Apollo-11 mission. These recordings represent a "worst-case scenario" for SAD, featuring far-field microphones, heavy ambient noise, hardware-induced harmonic interference, and highly variable speech densities.
The Limitations of Energy-Based Detection
Traditional SAD systems often rely on energy-based thresholds, assuming that speech is consistently louder than background noise. However, in naturalistic environments like the Apollo missions, this assumption fails. The energy distributions of speech and non-speech segments overlap significantly, making volume an unreliable metric for separation.
As shown in the figure above, the frequency of occurrences for normalized energy (plot a) reveals that speech and non-speech cannot be easily distinguished by energy alone. Furthermore, the duration of segments (plot b) varies wildly, with many files containing long stretches of silence punctuated by brief, bursty communications.
Proposed Methodology
To overcome these challenges, the researchers developed a multi-level, adaptive SAD system that focuses on the physical signatures of speech production rather than just volume. The architecture moves through several stages to refine the detection process.
1. Denoising and Level 1 Filtering
The raw audio is first cleaned using spectral subtraction. The system then applies Modulation Spectrum (MS) analysis. This feature captures the rhythmic patterns of human speech (typically in the 2-12 Hz range), which corresponds to the movement of human articulators. Because most background noise lacks this specific rhythmic "pulse," MS is highly robust to environmental hums and static.
2. Level 2 Filtering: Smoothed Hilbert Envelope
For regions where speech is suspected but not confirmed, the system analyzes the Smoothed Hilbert Envelope (SHE) of the Linear Prediction (LP) residual. This technique tracks the "excitation source"—the vibration of the vocal folds—providing a second physical signature of speech.
3. Adaptive Q-Factor Thresholding
The most novel contribution of this work is the Quality Factor (Q-factor). The system calculates this factor by comparing the energy of the top 20% of frames to the bottom 20%. This allows the algorithm to automatically categorize a file as Sparse, Balanced, or Dense in terms of speech content. The system then adaptively adjusts its SHE and duration thresholds based on this context, allowing it to "read the room" and maintain sensitivity even in muffled or occasional communications.
Experimental Results
The system was evaluated using the Fearless Steps Challenge 2019 corpus, which includes audio from iconic phases like the lunar landing and moonwalk. Performance was measured using a Detection Cost Function (DCF), which penalizes missed speech more heavily than false alarms.
- Proposed System: 7.35% DCF
- Challenge Baseline: 11.70% DCF
The multi-level approach significantly outperformed the baseline and maintained a better balance between false positives and false negatives than standard statistical models. This performance earned the team a 9th-place ranking out of 27 international submissions.
Conclusion
By combining features that describe both speech production (Modulation Spectrum) and the vocal source (Hilbert Envelopes) with an adaptive thresholding mechanism, the researchers demonstrated that robust SAD is possible even in the most degraded analog environments. Future work will focus on automating the threshold tuning process through machine learning and testing the system's generalizability to other naturalistic datasets, such as body-cam footage and public space surveillance.
Summary updated