Abstract: Thamar Solorio, Yang Liu. Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing.
2008.
Introduction
Diagnosing Specific Language Impairment (SLI)—a condition where children struggle with language acquisition despite having no other cognitive or neurological issues—is notoriously difficult in bilingual populations. Standardized diagnostic tests are largely designed for monolinguals, and there is a critical shortage of bilingual clinicians. Furthermore, bilingual children naturally engage in "code-switching" (alternating between languages), which can mimic or mask grammatical errors to an untrained eye or a standard diagnostic tool. This research aims to create an automated, statistical method to differentiate between typical bilingual development and SLI by focusing on underlying syntactic patterns.
Methodology
The researchers moved away from traditional word-based analysis, which can be biased by vocabulary size, and instead utilized Language Models (LMs) trained on Part-of-Speech (POS) tags. The methodology followed a four-step process:
- POS Tagging: Transcripts of children's speech were converted into sequences of grammatical tags (e.g., Noun, Verb, Adjective). This abstracts the data away from specific vocabulary and focuses on the structure of the sentences.
- 3-Gram Models: The team employed a standard 3-gram approach, where the probability of a tag depends on the two preceding tags.
- Dual Models: Two distinct LMs were trained: $M_T$ (trained on speech from Typically Developing children) and $M_I$ (trained on speech from children with SLI).
- Perplexity as a Metric: To classify a new speech sample, the researchers calculated its perplexity—a measure of how well a probability model predicts a sample—against both models.
Decision Rule: If the SLI model ($M_I$) showed lower perplexity (meaning it was "less surprised" by the speech pattern) than the TD model ($M_T$), the child was classified as having SLI.
Experimental Setup
The study utilized a specialized dataset of transcripts provided by speech pathologists:
- Typically Developing (TD) Group: 9 bilingual children (5 male, 4 female) aged 48–72 months. These children lived in homes where Spanish was spoken roughly 46% of the time.
- SLI Group: 2 bilingual children (both female) aged 53 and 111 months, previously diagnosed with language impairment through clinical evaluations in Spanish.
- Preprocessing: All transcripts were POS-tagged using a specialized bilingual tagger designed to handle Spanish-English code-switching.
Results
Despite the small sample size, the preliminary results were promising. The models correctly identified 7 out of 9 TD children and 2 out of 2 SLI children. The quantitative data reveals a clear trend in perplexity gaps between the two groups.
As shown in Table 1, TD children had a significantly lower average perplexity on the TD model (18.06) than on the SLI model (25.75). Conversely, the SLI children showed lower average perplexity on the SLI model (15.76) than on the TD model (19.95). The two misclassifications ($TD_4$ and $TD_9$) occurred where the perplexity scores were closer or favored the SLI model, suggesting these specific samples shared more syntactic similarities with the impaired group's statistical profile.
Discussion and Conclusion
By using POS tags rather than words, the model captures the "grammatical fingerprint" of the child. This is crucial because SLI is often characterized by specific syntactic deficits—such as dropping markers or incorrect word order—rather than just a limited vocabulary. Furthermore, because the tagsets for Spanish and English differ, the POS sequences inherently capture the points where a child switches languages. The results suggest that the way a child switches languages differs between TD and SLI groups, and statistical inference can pick up on these subtle markers.
The authors acknowledge the primary limitation of this study is the extremely small sample size, particularly in the SLI category. Future work will focus on scaling the data, testing the models across different language tasks (e.g., storytelling vs. spontaneous play), and evaluating whether these syntactic markers are universal or language-specific.
Summary updated