TL;DR
This paper explores how parents express concerns about potential autism spectrum disorder (ASD) in their children through online forums. They developed a machine learning approach to analyze these concerns and predict ASD risk, showing that coded signs are more effective than raw text for this prediction. This work highlights the value of parental narratives for early ASD screening and suggests the development of online tools to assist in this process.
What Problem Does This Paper Address?
Early identification of Autism Spectrum Disorder (ASD) is crucial for timely intervention and improved outcomes for children. However, the diagnostic process can be lengthy and complex, often starting with parental concerns that may not be immediately recognized or acted upon. While online communities offer a platform for parents to seek information and validate their concerns, the unstructured nature of these discussions makes it challenging to extract meaningful insights for screening. This paper addresses the need for a systematic way to analyze these online parental concerns to identify potential ASD risk, aiming to bridge the gap between parental intuition and professional evaluation.
Methods
The researchers employed a two-pronged approach: content analysis of online queries and machine learning for risk prediction.
- Content Analysis:
- Data Source: Queries from Yahoo Answers related to autism, Asperger's, or PDD were collected.
- Risk Categorization: A clinical expert, and later two experts, categorized the risk level of each query as low, medium, or high based on predefined rules related to ASD-specific and non-ASD-specific signs, developmental delay, and risk factors.
- Sign Identification: Another expert conducted a detailed content analysis to identify and categorize warning signs mentioned by parents. This involved both deductive (mapping to DSM-V criteria) and inductive (identifying other concerning signs) methods, resulting in a taxonomy of 12 domains and 72 subdomains.
- Machine Learning for Risk Prediction:
- Feature Engineering: Two types of features were used:
- Textual Data: Raw text of the queries.
- Coded Signs: Presence or absence of identified sign domains and subdomains.
- Classification Model: A linear classifier with a least-squares criterion was trained. For distinguishing between low-risk and medium/high-risk queries, a decision tree classifier was also developed.
- Evaluation Metric: The Area Under the Receiver Operating Characteristic (ROC) curve (AUC) was used to assess the performance of the models.
Datasets
- Yahoo Answers Queries: A dataset of 8681 English-language queries from Yahoo Answers submitted between June 2006 and December 2013, containing keywords like "autism," "Asperger," "ASD," or "PDD."
- Sampled Queries: A subset of 195 queries from parents suspecting ASD in their child was randomly selected for detailed analysis.
Key Insights and Ideas
- Parental Concerns are Validated: The study found that a significant majority of parental concerns expressed online were rated by clinical experts as medium or high risk for ASD, underscoring the importance of listening to parental intuition.
- RRBI and Language Dominance: Repetitive and Restricted Behaviors and Interests (RRBI) and language-related concerns were the most frequently reported signs by parents. This aligns with core ASD diagnostic criteria but also highlights the prevalence of language concerns even in non-ASD-specific contexts.
- Coded Signs Outperform Raw Text: Machine learning models that utilized coded signs (derived from expert analysis) performed significantly better in predicting ASD risk than models relying solely on the raw textual content of the queries. This suggests that structured interpretation of parental narratives is key.
- Age-Specific Patterns Emerge: While overall risk group distribution didn't vary significantly by age, language concerns were reported less frequently in older children, likely because parents of older children who haven't been diagnosed may have moved beyond the initial suspicion phase.
- Non-ASD Signs are Informative: The study noted that concerns related to domains not strictly defined as ASD-specific (like emotional and cognitive) were still prevalent and contributed to risk assessment, indicating a broader spectrum of parental worries.
Key Results
- Prevalence of Signs: RRBI (75.4%), language (61.5%), and emotional markers (50.3%) were the most prevalent concerns.
- Risk Distribution: Of the 195 queries, 18.5% were low-risk, 30.8% medium-risk, and 50.8% high-risk.
- Machine Learning Performance:
- Distinguishing high-risk from low/medium-risk queries:
- AUC using text alone: 0.67 (95% CI 0.50-0.78)
* AUC using coded signs: 0.82 (95% CI 0.80-0.86)
- Distinguishing low-risk from medium/high-risk queries:
- AUC using text: 0.54 * AUC using signs: 0.84
- Decision Tree Predictors: The decision tree model identified social, RRBI, communication, and motor concern domains as significant predictors of high ASD risk. Specifically, social problems increased the chance of high risk to 78%, and a combination of RRBI and motor delay led to a 100% chance of high risk.
Limitations and Future Work
- Lack of Clinical Validation: The study acknowledges a key limitation is the absence of actual clinical testing for ASD risk status for the children in the sample. Future work should involve validating the risk predictions against standardized developmental measures.
- Textual Ambiguity: Extracting signs from text can be challenging, requiring clinical inference, which can be subjective. Developing more structured formats for parents to input concerns and further testing the algorithm with new corpora are suggested.
- Generalizability: The study used data from Yahoo Answers, which may not fully represent all online parental discussions.
- Future Directions: The authors propose developing Internet-based screening systems that utilize parents' narratives through a decision tree questioning method. This could help automate ASD screening, reduce screening time, increase response rates, and improve accuracy, ultimately supporting worried parents and facilitating timely clinical evaluation. They also emphasize the need for parent education materials to help interpret RRBIs and manage concerns.
Summary updated