Introduction
In 2018, a simple audio clip went viral, dividing the internet between those who heard "Laurel" and those who heard "Yanny." This phenomenon highlighted a curious aspect of human auditory processing: our internal decision boundaries for speech perception are often remarkably close to one another, with little to no "dead zone" in between. While the Laurel/Yanny clip was treated as a viral fluke, it raised a deeper scientific question: how common is this phenomenon? Is human speech perception generally robust, or are we frequently on the verge of misinterpretation?
This research moves beyond anecdotal evidence to systematically quantify the prevalence of "polyperceivable" audio—clips that can be perceived as two or more distinct words by different listeners or under different conditions. By leveraging machine learning, the authors sought to estimate how many common English words can be rendered ambiguous through simple perturbations.
Methodology
To conduct a large-scale search for polyperceivable audio without the prohibitive cost of testing hundreds of thousands of clips on humans, the researchers developed a two-phase pipeline centered on an autoencoder proxy.
The Autoencoder Proxy and the $S$ Metric
The team trained a simple autoencoder on spectrograms of synthesized speech. The core hypothesis was that the autoencoder’s low-dimensional latent space could serve as a proxy for human "perceptual space." They introduced a metric, $S$, representing the "path length" of an audio clip’s latent representation as its playback speed is slowed down. A long, winding path in latent space suggests that the sound is undergoing significant structural changes in the eyes of the model, making it more likely to cross a human perceptual boundary.
Importance Sampling and Validation
Using the $S$ metric, the researchers performed "importance sampling" to identify the most promising candidates from a pool of 160,000 clips (generated from 10,000 words across 16 different voices using Amazon Polly). This allowed them to focus human testing on clips with the highest likelihood of being polyperceivable.
Human validation was conducted via Amazon Mechanical Turk, where thousands of listeners provided their perceptions of the sampled clips. The researchers then applied K-means clustering to the "bag of phonemes" from these responses to identify distinct perceptual "modes" for each clip.
Key Results
The study's findings suggest that polyperceivable audio is far more common than previously thought. The authors estimate that at least 2% of the top 10,000 English words can be rendered polyperceivable through simple playback speed adjustments.
As shown in the clustering data, many words exhibit a clear shift in perception as the playback speed is reduced. For example, the word "Settle" was frequently heard as "Civil," "Growing" as "Brewing," and "Bologna" as "Morning." The $S$ metric proved to be a highly effective tool for this search, increasing the "hit rate" for finding ambiguous clips by 3x compared to a random search.
Discussion and Implications
This research frames polyperceivable audio as a form of "human adversarial example." Just as tiny perturbations can cause a neural network to misclassify an image, minor changes in audio speed can push human perception across a decision boundary. This challenges the myth of the "dead zone"—the idea that there is a safe buffer between the sounds of different words.
Limitations and Future Work
The study primarily used linear slowdowns as perturbations. While effective, this does not perfectly mirror natural human speech, where slowing down involves changes in articulation. Future research could explore more "ecologically plausible" perturbations. Additionally, the 2% estimate is likely a conservative floor, as the researchers only manually reviewed a small subset of the most promising candidates.
Security Considerations
The ability to systematically identify these perceptual boundaries carries potential security implications. If an adversary can precisely edit a recording to be polyperceivable, they could potentially craft messages that are heard differently by different audiences, leading to misinformation or confusion. This remains a critical area for future investigation in the intersection of acoustics and security.
Summary updated