Introduction
In speech recognition, labeled data—where audio frames are precisely mapped to phonetic tags—is a critical but scarce resource. While the Switchboard Transcription Project (STP) provided high-quality human annotations, it covered only about 75 minutes of speech. This is insufficient for modern systems requiring thousands of hours of data. Semi-supervised learning (SSL) offers a path forward by leveraging small labeled sets alongside vast unlabeled corpora. However, traditional graph-based SSL methods often struggle with scalability, inappropriate loss functions for classification, and multi-class complexity.
Methodology: Measure Propagation
This research introduces a multi-threaded, graph-based SSL algorithm designed to scale to massive datasets, specifically the 120-million-frame Switchboard corpus. The core technique is Measure Propagation (MP).
Unlike standard Label Propagation (LP), which minimizes squared error, MP minimizes the Kullback-Leibler (KL) divergence between probability distributions. This approach is naturally suited for multi-class phonetic identification. The algorithm treats speech as a "low-dimensional manifold," assuming that mathematically similar audio snippets (graph vertices) should share phonetic labels.
To achieve the necessary scale, the authors implemented:
- Multi-threading: Using C++ and POSIX threads to parallelize label updates across CPU cores.
- Efficient Graph Construction: Utilizing the Approximate Nearest Neighbor (ANN) library and kd-trees to manage the complexity of similarity comparisons.
- Entropy Regularization: An entropy maximization term prevents the model from over-committing to labels prematurely, guarding against degenerate solutions.
Datasets
The project utilizes the Switchboard I (SWB-I) corpus, containing 2,400 telephone conversations. The "seed" data comes from the STP, a 75-minute subset with meticulous human-verified labels. The output of this project is S3TP (Semi-Supervised STP), a new dataset where the entire SWB-I training set is automatically annotated at the phonetic level using the MP algorithm.
Key Results
The MP algorithm demonstrated significant performance gains over both traditional supervised models and standard SSL baselines.
Even with zero unlabeled data, MP achieved 38.0% accuracy, outperforming a state-of-the-art supervised Multi-Layer Perceptron (37.3%). As the percentage of unlabeled Switchboard data increased, MP's accuracy improved consistently, maintaining a clear lead over the Label Propagation (LP) baseline. Notably, the system successfully processed a graph with 120 million vertices, representing a milestone in large-scale graph-based SSL.
Limitations and Future Directions
Despite its success, the implementation faced hardware bottlenecks. Sub-linear speedups during multi-threading were attributed to microprocessor cache performance issues caused by the massive, complex graph structure. Future work aims to move from frame-level classification to sequence-level SSL, which is more directly applicable to full-sentence recognition, and to explore articulatory labeling (modeling physical mouth movements).
Summary updated