Abstract
Identifying high-risk utterances in crisis counseling is critical for real-time intervention but suffers from a lack of expert-labeled data. This paper introduces a logic-driven indirect supervision framework that leverages abundant, coarse session-level risk labels to improve fine-grained utterance-level classification. By encoding clinical domain knowledge into differentiable logic rules using R-product t-norm relaxations, the model achieves a 3.5% F1-score improvement and a significant boost in recall for high-risk messages compared to standard multi-task learning baselines.
Introduction
In text-based crisis counseling, such as the 988 Lifeline, real-time identification of high-risk utterances is essential for supervisor prioritization and counselor feedback. However, training models for this task faces a significant "data bottleneck." Labeling individual messages for specific risk factors (e.g., "imminent risk" vs. "passive ideation") requires expert clinical knowledge and is prohibitively expensive. Conversely, session-level labels—where counselors tag an entire encounter as "high" or "low" risk—are abundant but coarse and noisy. This work explores how to bridge this granularity gap by using session-level signals to supervise fine-grained utterance classifiers.
Methodology
The proposed framework utilizes Logic-driven Indirect Supervision. Clinical guidelines are treated as logical constraints that link session-level and utterance-level data.
- Architecture: The model uses a shared RoBERTa-based backbone. It processes a chat session to generate message embeddings, followed by a Transformer encoder to capture conversational context. The model outputs both a session-level risk label and multi-label risk classifications for each utterance.
- Logical Constraints: Structural dependencies are defined using predicate logic. For instance:
- Joint Constraint: A session labeled "Lower Risk" cannot contain utterances with "High Risk" codes (e.g., "Attempt in Progress").
- D-Code Dependencies: Specific codes imply others; e.g., a "Prior Attempt" logically implies "Lifetime Ideation."
- Differentiable Loss: To integrate these rules into neural network training, the authors employ R-product t-norm relaxations. This converts discrete logic into a continuous loss function, penalizing the model when its predictions violate clinical consistency.
Data and Experimental Setup
The study utilizes two disjoint datasets from SafeUT, a regional suicide crisis hotline:
- Dataset U (Utterance-level): 425 sessions (~4,912 client utterances) labeled post-hoc by experts using a 9-dimensional risk coding scheme (D-codes).
- Dataset E (Encounter-level): 5,990 sessions labeled by the original counselors as simply "Higher" or "Lower" risk.
Crucially, no session exists in both sets, forcing the model to learn the relationship between the two levels of abstraction indirectly through the logical constraints.
Results and Discussion
The logic-driven model (MT+Rules) significantly outperformed baselines:
- Performance Boost: It achieved a 50.0 F1-score on utterance classification, representing a 3.5% improvement over multi-task learning and a 6.8% improvement over single-task models.
- Clinical Safety: Most notably, recall for high-risk sessions jumped from 24.8% (baseline) to 84.4%. In clinical triage, prioritizing recall is vital to ensure no high-risk cues are missed.
- Consistency: The model demonstrated higher logical robustness, violating clinical constraints far less frequently than purely data-driven approaches, making the output more trustworthy for human supervisors.
Conclusion
This research demonstrates that domain knowledge can effectively bridge the gap between coarse and fine-grained data. By enforcing logical consistency, models can learn more reliable representations from noisy, abundant session data. While the model is sensitive to hyper-parameter tuning for logic weights, its ability to prioritize recall makes it a promising tool for "human-in-the-loop" assistant systems in crisis counseling. Future work includes exploring larger language models and prompt-based architectures to further enhance performance.
Summary updated