Abstract
Substance use is a critical health issue among homeless youth, yet traditional screening methods are often burdensome and prone to bias. This study introduces a machine learning framework that leverages Facebook activity and a limited set of non-sensitive survey questions to predict substance use risk. By employing subword segmentation and multi-task learning, the model effectively handles the noisy and scarce data characteristic of this population. The results demonstrate that combining social media text with minimal survey data yields high predictive performance (up to 0.83 ROC-AUC), offering a scalable and less intrusive alternative for early intervention.
Introduction
Homeless youth in the United States, estimated between 1.5 and 3 million individuals, face disproportionately high rates of substance use, particularly involving high-risk substances like methamphetamines and opioids. Traditional methods for identifying at-risk individuals rely on intensive, self-reported surveys. However, these methods are often "onerous to collect," suffer from retrospective recall bias, and can quickly become outdated due to the transient nature of this population.
Given that approximately 80% of homeless youth use social media weekly—with Facebook being the primary platform—there is a significant opportunity to utilize this data for real-time screening. However, social media text from this population is often "noisy," characterized by typos and slang, and datasets are typically small, posing challenges for standard Natural Language Processing (NLP) models.
Methodology
The researchers developed a multi-step framework specifically designed to address the challenges of scarce and noisy data:
- Subword Segmentation: To handle typos and "Internet slang," the model uses Byte Pair Encoding (BPE). This breaks words into subword units, allowing the model to capture meaning even when words are misspelled or modified.
- Vectorization: The study compared various methods, finding that Distributed Bag-of-Words (D-DBoW), which learns document-level embeddings, provided the best representation for the text data.
- Multi-Task Learning: To mitigate the issues of small sample sizes and few positive cases for certain drugs, the authors employed multi-task learning. This approach allows the model to share supervision signals across related tasks (e.g., predicting different types of drug use simultaneously), improving overall performance.
- Multi-View Fusion: The framework integrates different data types using Late Fusion, which involves training separate classifiers for text and survey data and then ensembling their outputs. This method proved superior to early fusion for integrating high-level information.
Data and Experimental Setup
The study utilized a unique dataset consisting of approximately 135,000 Facebook posts and comments from 158 homeless youth collected over one year. This was paired with self-reported survey data regarding demographics and substance use, which served as the ground truth labels. The final analysis focused on 87 participants who provided both social media access and completed surveys.
Results
The proposed framework demonstrated significant improvements over baseline models. While standard models like BERT struggled with the noisy text, the combination of BPE and D-DBoW proved effective.
As shown in the results table, the Late Fusion approach (combining posts, survey answers, and comments) achieved the highest average performance. Specifically, the model was highly effective at identifying heroin use (0.815 AUC) and injection drug use (0.826 AUC).
The integration of just four non-sensitive survey questions regarding emotional stability and social life significantly boosted the model's accuracy, raising the ROC-AUC to 0.83 in the best-performing folds.
Discussion and Key Insights
A key finding of the study is that the "noise" in social media text contains valuable signals. Subword segmentation was crucial for navigating the linguistic chaos of the data. Furthermore, the researchers found that non-drug-related questions are highly predictive of substance use risk. Sentiment analysis also revealed that substance users tended to post more negative content (anger, disgust, sadness), whereas non-users used more positive language.
Limitations and Ethical Considerations
The authors acknowledge several limitations, including the small sample size and the potential for the model to learn false correlations. Ethical risks are also a primary concern; such tools must be implemented carefully to avoid stigmatizing youth or denying them services. Future work will focus on validating these findings on larger scales and addressing the privacy and transparency hurdles required for real-world deployment by non-profit agencies.
Summary updated