TL;DR
This paper introduces a weak-supervision framework to automatically build regional slang lexicons for specialized domains, specifically power-grid customer service. By combining "soft-label" bidirectional anchoring with active learning and LLM oracles, the researchers successfully extracted Yuxi dialect terms from noisy WeChat messages with perfect precision, significantly outperforming traditional "hard-gating" filtering methods.
What Problem Does This Paper Address?
In regions like Yuxi (Yunnan, China), power-grid customer service has shifted from formal hotlines to informal community social media groups (WeChat). However, customers rarely use standard Mandarin. Instead, they use regional dialects characterized by:
- Non-standard morphology: Frequent use of dialect-specific particles (e.g., ne, ga, ma).
- Semantic drift: Standard Chinese words taking on entirely different meanings. For instance, the phrase "old fire" (laohuo) in Yuxi dialect means a "severe fault," not a literal fire.
- Out-of-Vocabulary (OOV) issues: Dialectal expressions are poorly represented in the massive corpora used to train standard NLP models like BERT.
Manual lexicon construction is too expensive for every regional dialect, and standard NLP tools often misclassify these noisy, short messages. The authors aim to bridge this gap by creating a pipeline that bootstraps a domain-specific dialect lexicon with minimal human intervention.
Methods
The authors propose a four-stage weak-supervision framework:
- Candidate Generation via Bidirectional Anchoring: The system uses 16 curated dialect "anchors" (common particles or markers) and Pointwise Mutual Information (PMI) to find associated bigrams. Crucially, they use soft-labeling: instead of discarding terms that don't look like "power-grid" words immediately, they retain them and tag them with metadata, preserving the linguistic context.
- Semantic Classification: Candidates are scored using a dual-channel approach. One channel uses Sentence-BERT (SBERT) to measure semantic similarity to 65 power-domain seed terms. The second channel calculates a "wordhood" score based on how much the
jiebatokenizer resists splitting the candidate, which serves as a proxy for lexical fixedness. - Active Learning with Twin Oracles: An iterative loop queries an "uncertain pool" of candidates. They compared three query strategies (Margin, TD-CAL, and Random) and two types of oracles: a rule-based morphological matcher and a Retrieval-Augmented Generation (RAG) LLM (DeepSeek V4 Flash).
- Dual-Track LLM Evaluation: The final lexicon is evaluated by a higher-capability LLM (DeepSeek V4 Pro). To ensure this "LLM-as-a-judge" was reliable, the authors calibrated it against human expert annotations using Cohen’s $\kappa$.
Datasets
- WeChat Corpus: 11,177 cleaned messages extracted from Yuxi power-grid customer service groups over one year.
- Seed Sets: 65 power-domain seed words and 16 Yuxi dialect anchor markers.
- Knowledge Base: A 30-entry bilingual power-domain knowledge base used for the RAG-augmented LLM oracle.
- Reference Vocabulary: 839 terms from the Yuxi Dialect Vocabulary Compilation used for tokenization calibration.
Key Insights and Ideas
- The Power of Soft-Labeling: Traditional pipelines often use "hard-gating," throwing away anything that doesn't contain a domain keyword (like "electricity" or "meter") early on. The authors found that delaying this decision—the "soft-label" approach—retained 751% more candidates. This is vital because many dialectal markers of urgency or frustration don't contain domain-specific characters.
- Handling Semantic Drift: The paper highlights how critical it is to capture domain-specific meaning shifts.
As shown in Figure 1, a term like guihuo (literally "will-o'-the-wisp") actually signals customer anger and frustration in this context.
- Wordhood as a Signal: In Chinese NLP, where word boundaries aren't explicit, "segmentation resistance" is a clever feature. If a tokenizer refuses to break a string apart, it’s a strong statistical signal that the string is a meaningful lexical unit (a word) rather than a random fragment.
Key Results
- High Precision: All experimental configurations achieved a Dialect Precision (DP) of 1.0, meaning every term in the final lexicon was verified as a genuine Yuxi dialect feature with zero false positives.
- Coverage Gains: The soft-labeling approach expanded the candidate pool from 41 terms (under hard-gating) to 349 terms, a 751% increase.
- Oracle Performance: The RAG-LLM oracle was more "permissive," accepting ~89% of candidates, whereas the rule-based oracle was stricter (~47%).
- Human Calibration: The LLM evaluator showed perfect agreement with human experts on dialect identification ($\kappa = 1.0$) and strong agreement on power-domain relevance ($\kappa = 0.85$).
Limitations and Future Work
- Scale and Diversity: The study is limited to one dialect (Yuxi) and one domain (power grid). The authors suggest testing the framework on other major dialects like Cantonese or Sichuanese.
- Recall Metrics: While precision was perfect, the authors did not calculate "recall" (how many dialect terms they missed), as there is no exhaustive "gold standard" for this specific niche.
- Downstream Utility: The lexicon exists, but its actual impact on tasks like automated ticket routing or sentiment analysis remains to be proven in a live production environment.
- Model Circularity: The selection oracle and the evaluator belong to the same model family (DeepSeek). While calibrated against humans, future work should use entirely different model families to avoid potential shared biases.
Summary updated