Automated depression screening from recorded clinical interviews keeps running into one awkward fact. The public corpora are tiny, about 142 people in DAIC-WOZ, and adding voice and face signals to the transcript often makes models worse rather than better. The best reported DAIC-WOZ severity model uses text alone. We propose a different way to decide how much each signal should count. Build one patient-similarity graph per modality, so that people are linked when their speech, faces, or words look alike. Then weight each graph by how much clean group structure it contains, a quantity computed from the graph itself without any depression labels. Because the weights never touch the scarce labels, they cannot overfit to them. Because the graphs need only pre-extracted, optionally noised feature vectors rather than raw audio or video, the pipeline fits inside the privacy constraints that already govern these datasets.
The point is practical. Clinicians spend hours on interviews and questionnaires, and digital screening could triage that work. That only helps if the screen reports item-level symptom scores clinicians recognize, uses non-verbal cues without overfitting, and never requires pooling raw recordings. In plain terms, the claim is this: choosing modalities from the unlabeled structure of the patient population beats choosing them from the labels when labels are scarce, and the method survives the noise that privacy protection adds.
Core Claim
Multilayer network analysis studies graphs in which the same nodes are linked by several kinds of relation, each kind forming one "layer." It has a well-developed theory of when merging layers helps and when it hurts.
Chen et al. model each layer as shared signal plus layer-specific noise and merge layers by weighted sums. They prove a sharp phase transition: group structure is recoverable below a critical noise level and collapses above it. Spectral Clustering via Adaptive Layer Aggregation for Multi-Layer Networks chooses those weights automatically by maximizing a signal-to-noise criterion read off the merged graph's eigenvalues, a method the authors call SCME. They also find that Gaussian mixtures on the resulting embedding beat k-means.
None of this has been applied to clinical cohorts, where the "layers" would be modalities. The clinical literature, meanwhile, shows exactly the failure this theory predicts:
- Using Graph Representation Learning with Schema Encoders to Measure the Severity of Depressive Symptoms reports that its text-only graph model beats audio-video-text fusion on DAIC-WOZ and attributes this to overfitting at small sample size.
- Heterogeneous Network Approach to Predict Individuals' Mental Health finds that subsets of data types predict depression better than all of them together.
Our claim is specific. Let $A_\ell$ be the patient-similarity graph from modality $\ell$, for $\ell = 1, \dots, L$. Form the merged graph
$$A(w) = \sum_{\ell=1}^{L} w_\ell A_\ell, \qquad w_\ell \ge 0, \quad \sum_\ell w_\ell = 1,$$
where $w$ is chosen to maximize the spectral separation criterion of SCME. That criterion depends only on the graphs, never on PHQ-8 scores. A small regressor on the spectral embedding of $A(w)$ then predicts the PHQ-8 total and each of its eight item scores.
We predict two things. First, this label-free weighting matches or beats label-tuned fusion at full training size. Second, its advantage grows as the number of labeled training patients shrinks, because label-tuned weights are estimated from the same handful of labels they then overfit.
The prediction can fail cleanly, and the experiments are built to show it if it does.
Research Program
Privacy constraint. We adopt the constraint used by MLlm-DR, which keeps raw recordings away from the model by working only on pre-extracted audio-visual feature vectors. We tighten it in two ways:
- Local text encoding. The transcript is also turned into a fixed sentence-embedding vector, so no free text is shared.
- Optional local noise. Each participant's per-modality vector can be clipped to bounded norm and perturbed with Gaussian noise before it leaves the device. This gives a per-record differential-privacy guarantee, meaning any one person's data has a provably limited effect on what is shared.
Graph construction. From these vectors we build one k-nearest-neighbor similarity graph per modality: transcript semantics, acoustic prosody, and facial action units, all of which DAIC-WOZ and E-DAIC distribute in pre-extracted form. The graph uses every available participant, labeled or not. This is legitimate transductive use, because no labels enter graph construction or weighting.
Why the phase-transition theory is useful here. It turns privacy noise into a quantity we can reason about. Gaussian perturbation of features raises the structural noise in a layer. The theory of Chen et al. says each layer has a threshold beyond which its group structure disappears. Adaptive weighting should therefore down-weight an over-noised layer automatically instead of letting it corrupt the merged graph. We will measure where each modality's structure collapses as the noise scale increases. The result is an empirical privacy budget per modality, which is a deliverable in its own right.
Outputs. We report predictions at the item level because that is what clinicians use and what the LLM questionnaire-completion line of work targets. Are LLMs effective psychological assessors? and Delving into the Depths both fill in validated instruments item by item from text. The second documents that items a person never talks about, such as fatigue or sleep, default to zero. A similarity graph over voice and face gives such items a source of evidence that does not depend on explicit disclosure. This is the "hitchhiker" effect described in Global and individualized community detection in inhomogeneous multilayer networks, where weak layers borrow signal from strong ones. We test it directly on the somatic items.
A known risk: smoothing toward the mean. Graph smoothing tends to pull predictions toward the average. That would worsen the under-prediction of severe cases already reported for SGNN. We therefore report error on the high-severity tail separately from overall error.
Secondary analysis: subgroup extraction. We also apply the significance-based extraction method of Wilson et al., which finds groups that are unusually dense in only some layers. The question is whether severe patients form such a group within a subset of modalities. A positive finding would let a dedicated severe-case flag sit beside the regressor. A negative finding is still informative.
Second testbed. We repeat the core comparison on NetHealth, which adds a genuine social layer (SMS ties) alongside Fitbit sleep and step layers. Here at least one layer is a real network rather than a constructed similarity graph.
Baselines
Naive reference. Predict the training-set mean PHQ-8 total, and each item's training mean, for everyone. Every method must clear this, and it is especially informative on the high-severity tail, where shrinkage toward the mean looks deceptively good on overall error.
Unweighted averaging. Use the identical pipeline with all $w_\ell = 1/L$. This is the ablation that isolates our key ingredient. If adaptive weighting does not beat equal weighting on the same graphs and the same regressor, the multilayer theory contributes nothing here.
Concatenation. Join all per-modality vectors and fit ridge regression and gradient-boosted trees. This is the flat-feature baseline that Heterogeneous Network Approach to Predict Individuals' Mental Health found network methods outperformed on NetHealth, by 41% in precision and 64% in recall for depression. That paper reports no absolute baseline values.
Label-tuned stacking (strongest direct competitor). Train one regressor per modality, then learn modality weights by cross-validation on the training labels. It asks exactly our question, how much each modality should count, but answers it with labels. It receives the same features, the same inner cross-validation folds, and the same hyperparameter search budget as our method. We also run a variant that tunes a separate weight vector for each PHQ item.
Strongest published systems. We compare against two:
- SGNN, whose text-only DAIC-WOZ result is MAE 3.76 against a prior multimodal best of 3.88 (Using Graph Representation Learning with Schema Encoders).
- MLlm-DR, which reports CCC 0.72 on E-DAIC-WOZ (MLlm-DR). CCC is the concordance correlation coefficient, which penalizes both poor correlation and systematic offset from the true scores.
For context, the word-graph model of Burdisso et al. reports macro F1 of 0.84 for binary detection on the E-DAIC test set. No reported result was found on Lacuna for item-level PHQ-8 prediction on DAIC-WOZ, so item-level comparisons are against our own reimplemented baselines only.
Where code is available we rerun SGNN on our exact splits. Otherwise we quote its published number and say so.
What counts as beating them. Our method must:
- reach lower DAIC-WOZ MAE than 3.76, or at least match it while adding the item-level outputs and the privacy constraint that SGNN lacks;
- beat label-tuned stacking with confidence intervals that exclude zero at labeled training sizes of 50 or fewer; and
- not lose to stacking at full size.
Why not just…?
Why not just concatenate features and fit a regularized regressor? With roughly 142 to 275 labeled people and high-dimensional acoustic and visual vectors, concatenation is exactly the setting where SGNN's authors observed multimodal overfitting. Strong regularization might rescue it, so concatenation stays in as a baseline under the same tuning budget. If tuned ridge on concatenated features matches our method across the learning curve, that is a negative result we will report.
Why not just learn modality weights from the labels by stacking? This is the real threat. At full training size we expect stacking to be competitive, and we do not claim otherwise. Our argument is narrower: weights estimated from labels inherit the variance of a tiny labeled set, while spectral weights are estimated from all participants' unlabeled structure. The learning-curve experiment settles it. If stacking matches us at 25 and 50 labeled patients, the label-free premise is wrong for this domain, and the contribution reduces to the privacy-noise analysis.
Why not just use a text-only model, or prompt an LLM to fill in the questionnaire? Text-only models are strong, as SGNN shows. LLM questionnaire completion is attractive because it is item-level and needs no training, reaching a 55% hit rate on eRisk 2019 (Are LLMs effective psychological assessors?). Its agreement with experts, however, was around Cohen's kappa 0.28 on earlier eRisk sets and 0.0648 on eRisk 2021 (Delving into the Depths). It also misses undisclosed somatic symptoms, and in its strongest form it sends text to cloud services. We run a local open-weight model with direct prompting, since that paper found chain-of-thought prompting hurts small models, as an item-level baseline on transcripts. We compare it with our method specifically on the sleep, energy, and appetite items. If it wins there, the non-verbal layers are not paying their way.
Proposed Experiments
Data and splits. We use DAIC-WOZ and E-DAIC with their official splits, reporting:
- MAE for the PHQ-8 total;
- CCC for the PHQ-8 total;
- per-item MAE;
- MAE restricted to participants scoring 15 or above, the high-severity tail.
All results are averaged over repeated seeds, with bootstrap confidence intervals. NetHealth replicates the depression and anxiety comparison against concatenation and DMF-style recommender baselines.
Decisive check 1: learning curve. Subsample the labeled training set to 25, 50, 100, and full size, holding graphs and weights fixed, since they are label-free. Compare against stacking and equal weighting at each size. The hypothesis survives only if the gap over stacking widens as labels shrink.
Decisive check 2: are the weights meaningful? The learned weights should be stable across bootstrap resamples of participants. They should also rank modalities in the same order as each modality's held-out solo performance. Unstable or contrary weights would mean the spectral criterion is tracking something other than clinically useful structure.
Decisive check 3: privacy sweep. Increase the Gaussian noise scale on shared vectors, and record both each modality's collapse point and the degradation curve of each method. We predict adaptive weighting degrades gracefully, shifting weight toward the modality that remains structured, while concatenation and equal averaging collapse together once any one layer crosses its threshold.
Decisive check 4: tail and somatic items. If high-severity MAE is worse than the mean-prediction reference or than stacking, the smoothing concern is confirmed. In that case the subgroup-extraction flag becomes necessary rather than optional. If somatic-item error does not improve over the text-only LLM baseline, the hitchhiker argument does not hold for this data.
References
- Multilayer spectral graph clustering via convex layer aggregation.
- Spectral clustering via adaptive layer aggregation for multi-layer networks
- Using Graph Representation Learning with Schema Encoders to Measure the Severity of Depressive Symptoms
- Heterogeneous Network Approach to Predict Individuals' Mental Health
- MLlm-DR: Towards Explainable Depression Recognition with MultiModal Large Language Models
- Are LLMs effective psychological assessors? Leveraging adaptive RAG for interpretable mental health screening through psychometric practice
- Delving into the Depths: Evaluating Depression Severity through BDI-biased Summaries
- Global and Individualized Community Detection in Inhomogeneous Multilayer Networks
- Community Extraction in Multilayer Networks with Heterogeneous Community Structure.
- Node-weighted Graph Convolutional Network for Depression Detection in Transcribed Clinical Interviews