TL;DR
This paper introduces DARLA, a web application that automates the analysis of vowel formants in speech recordings, a crucial task in sociophonetics. By leveraging automatic speech recognition (ASR), DARLA significantly reduces the manual transcription bottleneck, enabling researchers to analyze large speech corpora more efficiently and gain insights into dialectal variation.
What Problem Does This Paper Address?
Sociolinguistics, particularly the subfield of sociophonetics, relies heavily on analyzing how speech sounds vary across different speakers and social groups. A key aspect of this is measuring phonetic features, such as vowel formants, which are characteristic of different accents. Traditionally, this involves manually transcribing audio recordings and then extracting formant measurements. This manual process is incredibly time-consuming and labor-intensive, especially for large datasets (web-scale corpora). While computational tools exist, they still require manual transcriptions as input, creating a significant bottleneck. The authors aim to overcome this by developing a system that can automatically transcribe speech and extract vowel formants without requiring any manual transcriptions beforehand.
Methods
The core of DARLA is an automated pipeline that takes an audio recording as input and outputs vowel formant measurements. The process involves:
- Automatic Speech Recognition (ASR): The system uses an HMM-based speech recognizer built with the CMU Sphinx toolkit. They trained acoustic and language models on a variety of American English corpora, including broadcast news and telephone conversations. A key innovation is the development of a dialect-specific acoustic model for Southern speech, trained on portions of the Switchboard corpus.
- Forced Alignment: After ASR, the system performs forced alignment to match the generated transcriptions to the audio at the phoneme level. This is crucial for pinpointing the exact location of vowels within the speech signal.
- Formant Extraction: The system then uses the formant extraction portion of the FAVE (Forced Alignment Vowel Extraction) code to measure formant values (specifically F1 and F2) for stressed vowels.
- Filtering: To improve accuracy, the system filters out low-confidence vowel tokens based on acoustic likelihood. They also filter out function words, unstressed vowel tokens, and tokens with high formant bandwidths, which can indicate unreliable measurements.
- Web Application Interface: DARLA is implemented as an online web application using HTML, JavaScript, CGI, and Ajax. This design choice eliminates the need for users to install software and allows for seamless updates. Users upload audio files, select dialect-specific acoustic models (Standard American or Southern), and specify speech type (free speech or word list). They also provide speaker information (ID and sex) for formant extraction.
- Output Generation: The results are emailed to the user and include scatter plots of vowel formants, raw formant data in a CSV file, and aligned ASR transcriptions in TextGrid format.
Datasets
The paper mentions training acoustic and language models on "a variety of American English speech corpora (broadcast news and telephone conversations)." They also specifically trained a dialect-specific acoustic model for Southern speech on "portions of the Switchboard corpus." For their case study, they used the audio of "US President Barack Obama's 2015 State of the Union address."
Key Insights and Ideas
- ASR is "good enough" for sociophonetics: The central insight is that current ASR systems, despite not achieving perfect word error rates, are sufficiently accurate for measuring key phonetic characteristics like vowel formants, which are essential for dialect research. This challenges the long-held assumption that manual transcription is a prerequisite for such analyses.
- Automating the bottleneck: DARLA directly addresses the laborious manual transcription step, which is the primary bottleneck in analyzing large speech datasets for sociolinguistic variation.
- Dialect-specific models improve accuracy: The paper highlights the benefit of using dialect-specific acoustic models (e.g., for Southern speech) to improve transcription accuracy, suggesting that tailored models can capture regional phonetic nuances better.
- Contextual information is valuable: The system outputs phonetic contexts for vowels, which is important because vowel shifts are often context-dependent. This allows researchers to analyze these relationships more deeply.
- Web-based accessibility: Providing the tool as a web application makes it easily accessible to researchers without requiring complex software installations, democratizing access to advanced phonetic analysis tools.
Key Results
- The ASR transcriptions for Obama's State of the Union address showed a 42% word error rate.
- The stressed vowel error rate was 13%, and for filtered tokens, it was even lower at 9%.
- The mean formants extracted from the ASR transcriptions were found to be similar to those extracted from manual transcriptions, with the largest discrepancies in less frequent vowels.
- The processing pipeline for a ~1-hour audio file took approximately 90 minutes to complete.
Limitations and Future Work
- Transcription Errors: While the system filters out low-confidence tokens, transcriptions can still contain errors, which may affect formant measurements. The authors plan to incorporate models of duration and acoustic features to further improve error identification.
- Limited Dialect Support: Currently, DARLA primarily supports Standard American and Southern English dialects. The authors plan to add support for other dialects in the future.
- Future Features: They are developing an interface for users to make corrections to transcriptions and receive updated formant measurements. In the longer term, they aim to expand beyond vowel formants to classify other dialect variables like rhoticity, nasality, and prosody. They also plan to improve the ASR error rate by incorporating state-of-the-art deep neural network technologies.
Summary updated