Is There Really a Category Here? A Calibrated Test Built on Face Clustering and Applied to Psychiatric Data

Psychiatry keeps asking whether its diagnoses are real kinds, like species, or cut points drawn on a continuum, like "tall." The usual tools cannot settle this. Clustering algorithms always return clusters. Supervised classifiers reach mediocre accuracy that could mean either "fuzzy boundary" or "no boundary." Variance-explained ratios describe the data without testing anything. Open-world face and person clustering in video faces the same statistical question, with one difference: there the answer is known. Different people are genuinely separate entities. Within one person, pose, expression, lighting and behavior vary continuously. This proposal uses identity-discovery data as a calibration bench for a model that separates discrete modes from continuous individual variation. It then applies the calibrated model to psychiatric cohorts to ask, with measured error rates, whether diagnostic categories or data-driven "biotypes" correspond to real gaps in the data.

In plain terms, we build a clustering model that is allowed to answer "there is only one group, and people just vary smoothly." We check that it gets the right answer on video of people, where the truth is known. Only then do we trust what it says about autism, mood and psychosis data.

Core Claim

The claim is that discreteness can be tested operationally, and that open-world identity data are the right place to validate the test. We call a structure discrete when the data contain density gaps that a smooth deformation of a single continuous population cannot produce. A model with two layers can implement this definition. The first is a nonparametric mixture, meaning a mixture whose number of groups is inferred rather than fixed and which can settle on a single group. The second is a smoothness-constrained continuous residual for each individual. The posterior probability of more than one mode is then a meaningful quantity. Its error rates can be measured on face tracklets and on within-identity variation before it is applied to clinical data.

The new contribution is not "cluster psychiatric data nonparametrically." Biotype studies already cluster. Mode-Aware Phenotype Profiling from Korean Clinical Reports already pairs Gaussian-mixture modes with a residual layer. What is new is a discreteness test with calibrated false-discovery behavior, validated where ground truth exists. We also add a within-cohort positive control. Acquisition site and document type are genuinely categorical nuisance variables, and the test must find them.

Where the Pieces Come From

The mechanism comes from identity discovery. From Face Recognition to Models of Identity uses a Hierarchical Dirichlet Process to let the number of identities grow with the data. It separates the question "is this the same entity?" from the question "what is its name?", and treats human tags as noisy and sometimes "unknown." We reuse the nonparametric identity layer and the idea that labels are noisy names attached to latent entities. We drop the video-specific context and face components. A Prior-Less Method for Multi-Face Tracking in Unconstrained Videos supplies the insight that Gaussian-process smoothing absorbs within-person feature drift, the continuous residual that would otherwise fragment one person into several clusters. We turn that post-processing step into a modeling constraint. Id-Free Person Similarity Learning and Video Indexing Using Face Appearance and Shot Transition Detection use density clustering and adaptive thresholds. These enforce hard partitions, which is the behavior our model must avoid when no partition exists. The Video Indexing pipeline was run on clinical autism therapy video but stopped at locating faces.

The clinical side supplies both motivation and constraints. Linked dimensions of psychopathology and connectivity found four continuous brain–symptom dimensions in the Philadelphia Neurodevelopmental Cohort rather than diagnosis-aligned groups. Breaking the Norm measured structural-MRI deviations along continuous axes, with depression and anxiety sharing one axis. Mode-Aware Phenotype Profiling reported that four "weak" autism modes account for 24.7% of phenotypic variance, and that one mode was driven by report format rather than clinical content. That last finding is the sharpest warning in the file: clusters can be real and still be artifacts of documentation. Deriving reproducible biomarkers from multi-site resting-state data shows that autism-versus-control classification falls to 67% under leave-site-out validation. This number is ambiguous between "noisy category" and "no category," and our test is meant to separate those two readings.

Research Program

The model writes each individual's embedding $z_i$, drawn from a pretrained face encoder, a connectivity vector or a text-derived profile, as

$$z_i = \mu_{c_i} + f(s_i) + \epsilon_i,$$

where $c_i$ is a mode assignment under a Dirichlet process prior, $\mu_c$ is that mode's center, $s_i \sim \mathcal{N}(0, I)$ is a low-dimensional continuous residual, $f$ is a smooth map shared across modes, and $\epsilon_i$ is isotropic noise. We place a Gaussian-process prior on $f$ with a lower bound $\ell$ on its lengthscale.

The lower bound is what makes the model identifiable. A sufficiently wiggly $f$ could fake any cluster structure. A smooth $f$ can bend and stretch one population into a curved or skewed shape but cannot open an empty gap. Gaps must therefore be explained by extra modes.

The output is the posterior probability $P(K>1 \mid \text{data}, \ell)$, reported as a curve over $\ell$ rather than a single number. When there is structure, the model also reports the recovered modes and the share of variation carried by $f$ versus by mode membership.

The calibration bench has three tiers. The first is synthetic, with known answers. It includes curved and skewed single populations, which should give "one mode." It also includes pairs of populations with a controlled gap, which should be detected as the gap widens. The second tier uses identity data. We take tracklet embeddings from the Music Video and Body-Worn Camera datasets of Lin and Hung. Pooling tracklets of different people should yield multiple modes. Pooling many tracklets of one person should yield one mode with a large continuous residual, despite heavy pose and expression variation. Degrading embeddings with blur and downsampling produces graded separation between identities, which lets us trace the detection-power curve.

The third tier is the clinical application. In the Philadelphia Neurodevelopmental Cohort we test both the 111 symptom items and the connectivity data. In ABIDE we test connectivity embeddings before and after site harmonization. If access can be arranged with the authors, we also test the Korean report profiles, working in their isometric log-ratio coordinates to respect the compositional geometry.

The clinical tier includes built-in positive controls. Before harmonization, ABIDE's 17 sites should appear as discrete modes. In the report data, report type should appear as a mode. A test that misses these controls has too little power to say anything about diagnoses.

Baselines

The naive reference point is a fixed-$K$ Gaussian mixture scored by its variance explained. This is the reading used in the autism report study. It cannot return "no structure," so it only anchors how misleading variance ratios are on our synthetic continua. We will report the variance fraction this method assigns to modes in pure single-population data. If that number sits near the 24.7% reported by Mode-Aware Phenotype Profiling, variance ratios carry little evidence about discreteness.

The strongest existing competitors for choosing whether clusters exist are standard model-selection tools. The first is Gaussian mixture selection by BIC, meaning the Bayesian information criterion, which trades fit against the number of parameters. The second is the gap statistic. The third is a Dirichlet-process mixture without a residual layer, which is essentially the identity layer of Castro and Nowozin applied to these embeddings. No reported result found on Lacuna for any of these as a discreteness test. The file reports their use for clustering, not calibrated false-discovery rates.

Fairness requires identical inputs. All methods receive the same embeddings and the same dimensionality reduction. They get the same candidate range of $K$ and the same inference budget, measured in wall-clock time and posterior samples for the Bayesian methods. Hyperparameters are tuned only on the synthetic tier and are then frozen for the identity and clinical tiers.

For the identity tier, clustering quality is compared against the weighted clustering purity of up to 0.92 that Lin and Hung report on the Music Video dataset. Our model need not beat that figure, since it is a test rather than a tracker. It should not fall far below it when identities are discrete.

The key ablation removes the residual layer while keeping everything else, which isolates whether the smooth residual is what prevents spurious modes. A second ablation removes the lengthscale bound, to show that an unconstrained residual destroys power.

Beating the baselines means two things measured on the synthetic and identity tiers. First, the model must have a lower rate of false "discrete" calls on curved or skewed single populations, including single-identity pools. Second, its power at equal separation must be no worse. Both are summarized by calibration error of $P(K>1)$ against ground truth.

Why not just…?

Why not just use BIC or the gap statistic to pick the number of clusters? This is the real threat, and the experiments settle it directly. Our expectation is that BIC with Gaussian components will split curved or skewed continua into several components, because one Gaussian fits them badly. Symptom distributions in community samples are typically skewed, with most people near zero. The single-identity pools in the identity tier are exactly such non-Gaussian single populations. If BIC matches our false-discovery rate and power there, the modeling contribution collapses. What remains would be the calibration bench plus the site and report-type controls, and we would report the clinical conclusions using BIC instead.

Why not just rely on supervised classification accuracy? The 67% leave-site-out accuracy in Deriving reproducible biomarkers is consistent with two situations. One is two overlapping groups. The other is a single continuum on which the diagnosis marks a threshold. Accuracy cannot tell these apart, so it does not answer the question; it is the ambiguity the test is designed to resolve.

Why not just adopt the mode-plus-residual reading of Mode-Aware Phenotype Profiling? That study fits modes first and treats the residual as whatever is left over. Nothing in that procedure can conclude that the modes should not exist. The authors themselves found that one mode reflected report format. Our model fits both layers jointly and is checked against a positive control for exactly that artifact. It therefore tests their conclusion rather than restating it.

Proposed Experiments and Decisive Checks

The first decisive check is on the identity tier. When tracklets from many sessions of one person are pooled, the model must report a single mode with $P(K>1)$ below 0.1. When tracklets from different people are pooled, it must report multiple modes, and its purity should approach the tracker figures above. BIC and the residual-free Dirichlet-process ablation should fail the single-person case more often. If our model also fails that case, it is not fit for clinical use and we stop.

The second check comes from the clinical positive controls. The model must detect ABIDE sites as modes before harmonization, measured by adjusted Rand index against site labels, and it must stop detecting them after harmonization.

Only if both checks pass do we interpret the substantive result. That result is the posterior over $K>1$ for autism-versus-control connectivity after harmonization, and for the Philadelphia symptom and connectivity data. For any modes the model does find, we ask whether they replicate across a discovery/replication split. The split mirrors the independent replication of 336 participants in Linked dimensions of psychopathology, where the clinical loadings replicated at $r = 0.59$–$0.85$.

We also check the continuous side. Wherever the model reports a single mode, the learned residual coordinates $s_i$ should recover the published Mood, Fear and Externalizing dimensions at least as well as the sparse canonical correlation solution does. A null result is informative too. A calibrated test that finds no diagnostic gap in data where it does find site gaps is direct evidence for the dimensional view, rather than a failure to find structure.