BanglishDepYT: Depression Detection in Native Bengali Social Media with Emotion-Preserving Preprocessing and Domain-Adaptive Pretraining

Ayon Roy, Sadman Sharif, Syed Shams IslamarXiv 2026 · arXiv

TL;DR

Researchers have developed BanglishDepYT, the first large-scale dataset for depression detection collected natively from Bengali social media. By introducing emotion-preserving preprocessing (EPH) and domain-adaptive pretraining (DAPT), they demonstrate that preserving informal linguistic features—like "Banglish" (romanized Bengali) and emoticons—is critical for performance, allowing a specialized 12M-parameter model to match the performance of multilingual models trained on vastly larger corpora.

What Problem Does This Paper Address?

Most NLP research for low-resource languages like Bengali relies on "silver standard" datasets: English text (e.g., from Reddit or Twitter) translated into the target language. While this provides labels, it creates a massive "translation gap."

Real-world Bengali social media is messy. Users frequently use Banglish (Bengali words typed in Roman script), emoticons, and emotional elongation (e.g., repeating characters like "আআআআ" to signal distress). Translators have no reason to include these features, so models trained on translated data never learn to recognize them. Worse, standard NLP preprocessing pipelines often treat these informal features as "noise" and delete them entirely. The authors found that standard cleaning reduces the "Banglish Survival Rate" to exactly zero, effectively blinding the model to the most expressive parts of a user's message.

Methods

The authors propose a three-pronged approach to bridge this gap:

  1. EPH (Emotion-Preserving Hybrid Preprocessing): Instead of aggressive cleaning, EPH converts emoticons into semantic tags (e.g., [CRY]), protects negation words (which are often on stopword lists), preserves Banglish tokens, and normalizes (rather than erases) character elongation.
  2. CESS (Composite Emotional Signal Score): A new diagnostic framework to quantify how much emotional signal a preprocessing method retains, measuring entropy, Banglish survival, and emotional lexicon preservation.
  3. DAPT (Domain-Adaptive Pretraining): Following the "Don't Stop Pretraining" philosophy, the authors took a pretrained Bengali model (SahajBERT) and continued its training on 163,217 unlabeled YouTube comments. This allows the model to learn the specific vocabulary and informal syntax of the target domain before it ever sees a depression label.
  4. LLM-Assisted Annotation: To scale the dataset, they used a locally deployed Qwen2.5-7B model to generate pseudo-labels for over 96,000 comments, achieving substantial agreement with human annotators ($\kappa = 0.611$).

Datasets

  • BanglishDepYT (New): Collected from YouTube mental health comment sections. It includes 191,000 raw comments, a cleaned corpus of 163,217 for pretraining, and 1,000 gold-standard samples verified by bilingual experts.
  • BSMDD: The existing benchmark for Bengali depression detection, which consists of translated Reddit/Twitter posts.
  • SentNoB & BanglaHate: Used as auxiliary datasets to test the limits of dataset fusion.
Figure 1: Percentage of comments containing informal features in BSMDD and the full Ban- glishDepYT clean corpus. BSMDD contains none; these features appear naturally in comments collected directly from YouTube. BanglishDepYT rates are computed over the full clean corpus, not the feature-enriched annotation sample (Section 3.4).

As shown in Figure 1, the translated BSMDD dataset contains zero emoticons, Banglish tokens, or negation words, highlighting why models trained on it struggle with real-world social media text.

Key Insights and Ideas

  • Preprocessing is a Model-Dependent Lever: For "bag-of-words" models like SVMs, EPH is a game-changer, providing a 15-point F1 boost. For transformers, the gain is smaller but still significant, likely because subword tokenizers can partially compensate for "noisy" text.
  • Domain Relevance > Model Scale: The domain-adapted DAPT-SahajBERT performed comparably to MuRIL, a massive multilingual model from Google. This suggests that for niche tasks in low-resource languages, targeted pretraining on modest in-domain data is a highly competitive (and cheaper) alternative to massive multilingual scaling.
  • The "Survival" Problem: Standard preprocessing is often "settled science" in NLP, but this paper shows it can be a structural failure for informal text. If your preprocessing returns an empty string for a highly emotional Banglish comment, no amount of model architecture can fix it.

Key Results

  • Preprocessing Impact: On the native YouTube corpus, EPH raised the SVM F1 score from 0.7062 (standard cleaning) to 0.8570.
  • DAPT Success: DAPT-SahajBERT achieved the highest mean F1 (0.8161) and MCC (0.6272) among all evaluated transformers, significantly outperforming the SahajBERT baseline ($p = 0.0013$).
  • Scaling via LLMs: Semi-supervised training using LLM-generated pseudo-labels improved the Area Under the Curve (AUC) from 0.8499 to 0.8731 ($p = 0.044$), proving that LLMs can effectively scale annotation for sensitive mental health tasks.

Limitations and Future Work

The authors acknowledge several constraints:

  • Small Evaluation Set: The primary comparisons rest on a 150-comment test set. While seed-level results are robust, a larger "gold" test set is needed for definitive confirmation.
  • Contamination Risk: The DAPT corpus included the unlabeled text of the test comments. While no labels were seen, this "exposure" might slightly inflate results.
  • Demographic Bias: The data is limited to YouTube users engaged with mental health content, which may not represent the entire Bengali-speaking population (e.g., older or rural speakers).
  • Future Directions: The authors suggest expanding the gold-standard labels to 10,000+ samples and extending the EPH/CESS framework to other South Asian code-mixed languages like Hinglish or Tamil.

Summary updated

Abstract

Depression affects millions of Bengali speakers across South Asia, yet automated detection systems have relied almost entirely on datasets translated from English. Such datasets cannot capture the emoticons, romanized Bengali words, and elongated spellings that characterize real expressions of distress. We introduce BanglishDepYT, a Bengali depression dataset collected natively from YouTube, comprising 191,000 unlabeled comments and 1,000 samples verified through dual bilingual annotation. We propose EPH (Emotion-Preserving Hybrid preprocessing), which converts emoticons to semantic tags, preserves Banglish tokens, protects negation words, and normalizes emotional elongation. A key finding is that preprocessing choice substantially affects performance on informal text: standard cleaning reduces the Banglish Survival Rate to zero (SVM F1: 0.7062), while EPH preserves signals (CESS: 0.9996 vs. 0.6694) and raises SVM F1 to 0.8570, an improvement of 0.1508. We further apply domain-adaptive pretraining (DAPT), continuing SahajBERT pretraining on unlabeled in-domain comments. DAPT-SahajBERT achieves the highest mean F1 (0.8161) and MCC (0.6272), significantly outperforming the baseline (F1: 0.7805, p = 0.0013, d = 1.450) and matching MuRIL (p = 0.589). These results suggest that targeted domain pretraining is a competitive alternative to large-scale multilingual pretraining. LLM-assisted annotation using Qwen2.5-7B achieves substantial human agreement (kappa = 0.611), enabling pseudo-labeling of 96,071 comments. Semi-supervised training improves AUC from 0.8499 to 0.8731 (p = 0.044), offering a scalable path to expanding annotation. All code, models, and datasets are publicly released to support reproducible research.
BibTeX
@misc{roy2026banglishdepyt,
  author = {Ayon Roy and Sadman Sharif and Syed Shams Islam},
  title = {{BanglishDepYT: Depression Detection in Native Bengali Social Media with Emotion-Preserving Preprocessing and Domain-Adaptive Pretraining}},
  year = {2026},
  eprint = {2611.01633},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.01633},
}
Figure